At the simplest level, my role as a researcher is to act as a conduit between our clients and those they seek to serve. I ask questions. I listen to answers. I report back, often with analysis, interpretation, and advice. Then, with these insights, our clients are better equipped to make better decisions.
Of course, the quality of the response is only as good as the quality of the question. Questionnaire design is a topic we cover with some regularity in Tuesdays with Coleman because it is so fundamentally important to the research process. In that tradition, here are five tips to keep in mind when designing a survey.
1) Structure the conversation like a funnel, starting broadly and ending narrowly. The key here is that we are trying to avoid, or delay, planting ideas in the mind of the respondent. For example, we wouldn’t want to start a survey with a series of questions about bananas before then asking, “What is your favorite fruit?” because doing so would increase the likelihood of bananas being answered as their favorite simply because we had initially focused their attention on bananas. Instead, it would be better to ask our general questions about fruits before moving on to more pointed questions about bananas.
Structure questionnaires like a funnel, starting broadly and ending narrowly. (Photo credit: Shutterstock.com)
2) Separate quantitative research from qualitative research. To clarify:
- With quantitative research, we are quantifying (i.e., counting) levels of awareness, perception, taste, behavior, etc. among a large sample size of respondents.
- With qualitative research, we are primarily collecting data in the form of words rather than numbers. In-depth, open-ended feedback is gathered from a smaller set of respondents.
These two forms of research work very well in tandem, but neither is very effective at replacing the other. While throwing an open-ended question into a quantitative survey may feel like it will be an easy and efficient solution, the results are often underwhelming, with responses that are shallow, vague, scattershot, or hard to interpret. Moreover, open-ended questions are a common point of frustration for quantitative respondents, leading to dropouts or disengagement.
3) Use randomization or rotation instructions to minimize test order bias. If we were collecting opinions on ten fruits, would it make a difference if apples were always asked about first and watermelons were always asked about last? It might. For example, a respondent’s feedback might get more discerning as they build up points of comparison. Therefore, in question series and answer series, it can sometimes be helpful to implement instructions that mix up the order.
Photo credit: Shutterstock.com
4) Talk like a normal person, not an industry insider. This is the Coleman Insights principle known as “Inside Thinking vs. Outside Thinking.” “Inside Thinking” is an inside-the-boardroom perspective, where you think that everyone is as knowledgeable and as passionate about your industry as you are. Meanwhile, an “Outside Thinker” tries to get out of that bubble and put themselves in the shoes of the consumers they are trying to serve. When crafting a research questionnaire, don’t use industry jargon that your target audience may not understand, and don’t probe highly nuanced issues that your target audience is unlikely to have an opinion on.
5) Be efficient. If the respondent feels frustrated by what they perceive to be an overly long questionnaire or overly wordy questions, they may reduce the quality of their feedback or quit the survey altogether. Therefore, we want to balance accomplishing our goals with respecting their time. From a macro perspective, that means we should prioritize goals, consider actionability, and be ok with leaving some items out of the questionnaire. From a micro perspective, that means we should write questions with as few words as possible without sacrificing meaning, avoid devoting two questions to a topic that can be covered in one, and utilize skip patterns and auto-coding to speed people past questions that they don’t need to answer.
It’s always helpful to get in touch with your target audience. Hopefully, these suggestions will help you get the most out of those opportunities.
