Hi, my name is David, and I obsess over research questionnaires.
“Hi, David. What exactly are you obsessing about?”
I think about the coding that goes into the data file. What if there’s a typo? What if the answer key is incorrectly ordered? What if there’s a mindless copy and paste and the code we meant to assign an answer doesn’t match the code that was captured in the data file? What then??
“Are these the only things keeping you up at night?”
Besides my dogs? No, I also worry about what devices respondents use to take the surveys. We’d like to think they’re all using a shiny 48-inch OLED monitor, but let’s face it–lots of them are taking it on a small phone with a cracked screen! What if the survey isn’t optimized properly for them??
“That seems like a lot of pressure.”
Oh, it is. It’s been more than 16 years since my last on-air shift, and I still have that off-the-air alarm dream. But more often than not, I think, did I check to make sure that the 5-point on the scale says “love it” and the 1-point on the scale says “dislike it a lot?” Did the introduction for that section say “click the button on the left” when the button is actually at the bottom of the screen when you look at the survey on a smartphone?
In this example from an entertainment study, respondents are asked to read the statement on the left and right, but in this mobile view the statements are on the top and bottom.
Are the questions being asked in such a way that will truly get the unbiased answers we’re seeking???
“Well, are they David?”
(takes a deep breath) Yes. Of course they are.
“And why is that? Doesn’t everyone do that?”
(icy cold stare followed by sarcastic eye roll)
“I see.”
It’s so important to get the right people. I’ll dream about how important the sample is.
So, what should I do about this obsession?
Drawing by Charles Barsotti
“Do your clients worry about this kind of thing? Are they concerned these kind of mistakes will be made and things will be overlooked?”
No.
“And why is that?”
We do it so they don’t have to worry.
(silence, followed by pondering)
So, you uh…think I should…
“Keep obsessing?”
Yes.
“Yes.”
Have a good day.
“Bye, David.”
