As you would probably expect, the thing everyone wants to know with every research study is, what are the results and what do they mean? But there’s a saying in the data business that should be considered in the development of any questionnaire:
“Garbage in, garbage out.”
In short, the quality of the output is determined by the quality of the input. And that starts with questionnaire development. A great deal of time at Coleman Insights is spent behind the scenes thinking about things such as how respondents will see or hear questions, which order they should go in, and so on. Whether we’re writing a complex survey for a media brand or you’re creating a simple survey in SurveyMonkey to send to your database, there are a fundamental set of procedures to consider. Here are three we keep in mind:
#3: Ask the most important questions in the beginning.
The beginning of a survey is when the respondent is most engaged. Focus on the most important issues facing your brand as close to the beginning of the survey as possible. If you’re a podcaster, you may want to ask listeners how familiar with and interested they are in the guests you plan to talk to in upcoming episodes. While that’s informative, it’s not as important as learning, and tracking, how many podcast listeners are familiar with your brand, and of those, how many listen regularly. The appeal of your guests is irrelevant if few people are even aware that your podcast exists.
#2: Don’t ask qualitative questions in a quantitative project.
Over the past few years, we’ve become more intentional about discouraging verbatims in perceptual studies. While there are a few exceptions, it’s like trying to paint a house with a hammer; you’re using the wrong tool for the job. In a qualitative study, like a Campfire Online Discussion Group or 20/20 Focus Group, an experienced moderator can probe participants for more details. A question like “Why are you listening to (radio station) less than you did a year or so ago” can be insightful in this type of research. However, an answer like “Because I listen to streaming more” may be interesting, but it isn’t actionable. With qualitative research, we have the opportunity to dig deeper to find the reasons why the consumer is listening to streaming more. Did your station do something to cause them to listen less? Was there something about a particular streaming service that attracted them? Or, did they get a new smartphone and haven’t downloaded your app yet? Or perhaps they didn’t know they can ask their new smart speaker to play your radio station?
That same listening momentum question can still be effective in a quantitative study, like a Plan Developer. Instead of an open-ended verbatim answer to the question, it is more effective to specifically test theories you have as to why they are listening less and give the respondent some choices. For example:
- The quality of the music has declined
- The station plays the same songs over and over
- The personalities on WAAA talk too much
- Big Bob left the WAAA Morning Show
- WAAA is playing too many commercials
- You are listening to (X) more
- Other (SPECIFY)
The result is likely to be much more actionable, which brings us to:
#1: Ask questions that will provide actionable information.
Other than satisfying intellectual curiosity, there is no value in learning information that you can’t do anything with. For example, you may be curious what percentage of your listeners listen to your station on a radio vs. your app. However, what are you going to do with this information when you have it? If your goal is to get more people to use your app, you should consider a Campfire study to find out what people like and don’t like about it. Once you have the results, market what users like about the app and be sure to fix the issues that bother them.
