I was recently cleaning out the Coleman Insights archives and came across a column I wrote for Radio & Records in 1999. (Step in the DeLorean and check it out on Page 96 here.)
In that article, I described four problematic scenarios my colleagues and I often encountered when radio station personnel discussed research with us. What struck me as I read the column—beyond how much younger I looked in the photo that accompanied it!—is the frequency with which we still encounter many of these scenarios. Sure, the ways listeners consume audio and the dynamics of the radio industry have changed dramatically over the past 18 years, but the fact that the points raised in my column still resonated really caught my attention.
Thus, in this four-part blog series, I’ll revisit those scenarios with some updates. This week, we’ll focus on Scenario 1 – The Methodology Focus.
There is absolutely nothing wrong with questions about research methodology. In fact, any manager who does not understand the methodology his or her research company is using—including the benefits and drawbacks of each element of that methodology—is selling his or her station short.
The problem with this scenario stems from where it starts. If you are dealing with a credible firm, they will propose a research program that is customized to your specific issues, rather than force a “boilerplate” 400-person perceptual study down your throat. While many firms (including ours) will have their own methodological approaches, the good ones will adapt their ways of doing things to what you need to accomplish with your research.
Instead of focusing on the technical merits or drawbacks of a potential research partner’s methodology, you should be trying to answer more important questions about them, such as:
- “Have they demonstrated a clear understanding of what I am trying to accomplish through research?”
- “How is their track record with helping stations in similar situations?”
- “Do they have an agenda or will they be straight with me about what the findings say?”
- “Will they provide me with data or will they go further and help me interpret the numbers and make recommendations on how I should proceed?”
- “To what degree will they help me implement changes based on the research?”
Believe me, methodology is important. There is enough research out there based on poor design, dubious samples and weak quality control worthy of multiple columns dedicated to that subject on its own. However, if the relationship with the research company is built on your feelings about their methodology and not on whether they truly understand your situation and have the knowledge to help you, the odds of both parties ending up disappointed are very high.
Why invest in research when ratings already look great? Next week, we’ll focus on Scenario 2 – We Have Great Numbers.
