Survey Design and Analysis

Bad surveys mislead. Principles, types, analysis.

Why survey design matters

Bad surveys produce misleading data. Good surveys produce actionable insights. Design rigor separates surveys that inform strategy from surveys that mislead.

Principles

• Specific research question first.
• Sample size statistical power.
• Question wording neutral.
• Question order considered.
• Response options balanced.
• Pilot test before full launch.
• Avoid leading questions.
• Avoid double-barreled questions.

Question types

• Closed (multiple choice, Likert).
• Open-ended (free text).
• NPS / CSAT.
• Conjoint (preference trade-offs).
• Van Westendorp pricing.
• Ranking questions.

Analysis

• Statistical significance.
• Subgroup analysis.
• Correlation vs causation.
• Open-text coding.
• Cross-tab analysis.
• Visualization for stakeholders.
• Synthesis and insights.

Advanced playbook

• Research question documented first.
• Sample size pre-calculated.
• Pilot test before full launch.
• Survey design reviewed.
• Subgroup analysis planned.
• Open-text coding methodology.
• Statistical significance reporting.
• Cross-tab analysis.
• External research vendor where appropriate.
• Annual survey program review.

Mistakes

• Research question absent.
• Sample size insufficient.
• Leading questions.
• Double-barreled questions.
• Pilot test skipped.
• Subgroup analysis absent.
• Open-text not coded.
• Significance not reported.
• External vendor not considered.
• Annual program review skipped.

Checklist

• Research question documented first
• Sample size pre-calculated
• Pilot test before full launch
• Survey design reviewed
• Subgroup analysis planned
• Open-text coding methodology
• Statistical significance reporting
• Cross-tab analysis
• External vendor where appropriate
• Annual survey program review

Part of the Marketing Research series.