Seven Steps to Survey Success (or ‘How to get unstuck on Methodology’) Professor Steven Litt
1. First: carefully consider whether the specific type of info/insight you wish to collect, and sample from whom you wish to collect it, is better suited to Qualitative research (ie 1 on 1 in-depth interviews, or Focus groups- for which a separate waiver is needed!) than Quantitative (such as a survey)
2. If you ARE in a situation that calls for doing a survey, familiarize yourself with the 3 basic Building Blocks of a survey, the three distinct question-response types. There’s no ‘Swiss army knife’ question-type for Quantitative research; most well designed surveys involve all 3 question types, since their strengths (and ‘fit factors’ for each) vary:
3. Now consider the survey medium – ie the venue through which the info will be collected. A key consideration is the (permission-approved) assets at hand (eg “I have permission-approved access to their email addresses” or “I have the faculty permission to have them fill out this survey in class”), however, an ‘available’ way to access survey information may not be the best way to maximize Sample participation, response detail, response honesty, etc.
4. Once designing a survey, if the entire process seems too ‘random/hit & miss’ or daunting, consider ‘chunking’ the work into steps. A helpful early step? Create a Matching Table; this requires you to check that you have indeed laid out all the specific Research Objectives for the study, unveiled key Hypotheses, and identified the need to compare Sample groups (segments). After you do all of this, start to write the specific questions that will extract the responses to fulfill your Research Objectives, prove or disprove the Hypotheses, identify the segment to which a Sample member belongs. Surveys are especially prone to ‘scope creep’; if you skip this step, you run the risk a survey will ‘expand’ uncontrollably and end up with many questions that are ‘off topic’ or outside the intended original scope of the study:
5. After you create a Matching Table, sequence the question-responses in a way that works best for a respondent; an ideal sequence should feel logical to them, help them start to engage with the survey, let them respond in more detail, and build the trust and rapport necessary to extract more honest, complete responses. Sequencing a survey:
6. If you are concerned respondents will not participate, or will not complete the survey once they start, invest some time considering that there are different types of Survey nonresponse, and there are different tools to address each. Does your survey, as designed, make best use of these different tools to combat each of the different types of Nonresponse? Reducing Survey Nonresponse
7. A critical next step is a beta test. Have several people you trust try your draft survey as AVATARS ie have them complete it and give you candid feedback as they role play different types/segments of respondents. This stage usually exposes survey flaws (eg The responses available in Question 4 don’t describe their reality; The wording in Question 5 is unclear; Their age bracket is missing in Question 9; etc). All flaws must be addressed before fully fielding the survey.