Recently a great, long-time client of ours disclosed to me that they fielded a research project with another company. This is not unusual; we often encourage our clients to get other perspectives on challenging situations.
In this case, however, the client told me that they were no more than two or three PowerPoint slides into the presentation when they realized something was amiss. There were numerous red flags in the data the other company began presenting that made our client question the credibility of the entire study. Before the presentation was even over, our client decided to set aside the research project.
I must admit that a part of me was pleased that this experience provided our client with a strong reaffirmation of the quality of the data they receive from us. As an advocate for research in the designing of strategies for media properties, however, I was angered that our client was so badly let down and how such an experience could taint the reputation of the research industry field in general. In addition, I was reminded how important it is that media companies understand as much as possible about what goes into producing high quality data for use in making decisions about branding, positioning, marketing and content execution.
At the heart of a good research study is that the respondents who participate in it appropriately represent the population you are trying to measure. In the “old days” this was relatively simple because we could easily reach virtually the entire population by conducting interviews on the phone. Today, when we conduct most of our interviewing online (although continue to conduct telephone interviews to reach some segments of the population), the process has become vastly more complicated.
This is why nine of my Coleman Insights colleagues—including representatives from our Integr8 Research subsidiary—and I spent three days north of the border a few weeks ago. The headquarters of our primary fielding partner is in Canada and we visited with their staff for what I like to think of as The Great Data Quality Summit.
The Coleman Insights and Integr8 Research teams with our fielding partners in Canada
We tackled important issues with our fielding partner, including improving security to prevent hackers from infiltrating our surveys, enhancing analysis tools to identify and remove respondents who are not who they say they are and steps to improve the survey experience for respondents. Our meetings were an important reminder for our staff and our fielding partner that the process of collecting research data online—and correctly integrating it with telephone survey data—is a challenging and evolving one and only by staying on top of developments in internet-based research can we ensure that we continue to deliver high quality data to our clients.
Conducting research online requires more than buying access to an online panel—even a so-called “A”-graded panel—and sending survey invitations to its members. Getting the “right” people in surveys who represent the population you are trying to measure requires knowledge about the panel assets being used, extensive experience in survey design, advanced analytical tools and dedication to doing the work required to produce high quality data.
We had nearly half of our staff spend three days in Canada to make sure we are on top of our game. If you are a Coleman Insights or Integr8 Research client, I am confident that you will see the benefits of our ongoing efforts to do things the right way. If you work with another research company, I urge you to learn as much as possible about how they gather, analyze and tabulate the data they deliver to you.
While we don’t foresee challenges getting any easier, we do feel accountability is imperative to ensure your study accurately guides your brand’s strategy, now and in the future.
