Every question and issue does not require a Coleman Insights-level research study. We encourage our clients to conduct smaller-scale surveys to solicit feedback from their consumers. We’ve all seen self-serve questionnaires, hosted on platforms like Google Forms or Survey Monkey, from companies on whose mailing lists you’ve landed. As you might expect, the consistency of surveys like this varies widely. Our SVP/Research Operations David Baird tackled this issue in the Tuesdays With Coleman blog “Three Best Practices of Questionnaire Development.”

Recently, I was served a Facebook ad from a city government. The hook was “We want to hear from you! We are polling the community to learn about priorities for next year’s city budget.”

A city government posted a survey via Facebook to solicit opinions from the community on budget priorities

I was a little concerned that the city was using a Facebook poll for budget priorities, but I respected the community reach out and was game. Sure, why not.

Question 1 asked how important it is that the city “provide each of the following services.”

There were 33 choices. Providing a safe reliable bus system, Quality downtown parking, Reducing traffic accidents and congestion, Providing parking enforcement, Quality sidewalks, Maintaining city streets, Adequate EV charging stations, Providing bike lanes, High quality spaces where people live, work, and relax, Well maintained infrastructure (water, sewer, electric/gas), Affordable housing options, Preventing fires through public education and safety inspections, Maintaining a clean downtown, Responding to community needs (fire, police, 911), Preparing for disasters (earthquakes, hurricanes, pandemics), Quality community centers, Environment that promotes diversity and inclusion, Quality city entertainment venues, Opportunities to celebrate, connect, and contribute to creative and cultural ecosystem, Preserving local history, Providing job opportunities, Supporting small business, Supporting equity and minority and women-owned businesses, Youth skill development opportunities, Protecting natural resources, Engaging with and reaching out to the community, Making it easy to report issues and make service requests, Making it easy to pay bills and fees, Supporting actions that may reduce energy bills, Reducing energy consumption and supporting renewable energy use throughout the community, Giving residents a chance to express their views before making budget decisions, Using emerging technology and data to improve city services, Offering quality garbage and recycling collection, and Giving residents the information they need about recycling in your neighborhood.

Did you really read all 33 choices? Don’t feel guilty if you didn’t. Most of the survey respondents didn’t read them either.

The next question asked about my satisfaction with each of these 33 services, followed by a verbatim question: “Anything else you’d like included in the budget?”

Oh, my goodness.

The first question didn’t ask me to rank which services were most important, just which ones I felt were important and unimportant. But as I read them, I couldn’t help but compare and contrast. What does “quality” mean to them? It could mean something different to me. And are they really asking if I think pandemic preparation and the fire department are important?

Call me crazy, but I rated disaster preparation as “Extremely Important.”

I chuckled when I got to the verbatim question, because I had to scroll and scroll and scroll to see what was covered already.

There are several ways the survey could have accomplished what the city was going after. It may have included wording the questions differently, reducing the number of answers, grouping the answers into categories, and actually ranking priorities – while that seems to be the intention of the study, unfortunately the survey design may not provide the clarity the city seeks.

When designing a questionnaire, always consider it from the vantage point of the one taking the survey. Is every question clear? Is every potential answer necessary? Will the design lead to answering the main questions/issues that inspired the survey in the first place?

If not, go back to the drawing board. The results are always only as good as the design.

If you’ve seen a Coleman Insights Plan Developer perceptual study, you may recall the term “Cume Conversion Rate.” Put simply, it is the percentage of Cume that a radio station converts into P1 listening. While success can vary based on factors such as the number of stations in a market, our typical Cume Conversion Rate benchmark is 40%. Meaning, if a radio station converts at least 40% of its Cume to P1, it is generally satisfying its listeners. Although this example applies to radio, this conversion is important to nearly every business. A flood of listeners checks out your podcast, but do they come back for more episodes? Lots of people may come into your restaurant, but are you earning their repeat business?

While it is never the sole answer to a burning question, it never ceases to amaze how often Cume Conversion Rate plays a key role in our findings.

In this example, Station A and Station B surpass our benchmark for Cume Conversion Rate, converting 58% and 45% of their Cume to P1, respectively

When a radio station is underperforming in the ratings and seeks to understand why, a high Cume Conversion Rate can be an indicator that the station is doing many things right with the audience it has. But its Cume may be too low, indicating a potential awareness challenge in the market.

We may see a similar scenario, where a station has high Cume but lagging numbers. A low Cume Conversion Rate may indicate that, while the station’s brand brings audience in, the product is misaligned in some way that is affecting the performance.

A low Cume Conversion Rate could be the result of several underlying issues that span a Plan Developer study. Is the station playing music that’s out of sync with market tastes? Are the personalities unknown or not making an impact? Has it not developed any images beyond its Base Music or Talk position and is thus seen as a utility? Has a station’s Cume Conversion Rate declined over time, and what does that tell us?

These are just a few examples of questions we seek to answer.

Cume Conversion Rate is a “what,” not a “why,” but insights culled from perceptual data gathered in a study lead us to the why.

It is those special instances when a station pulls it all together – high Awareness, delivering a largely appealing product aligned with the brand, along with deeply developed images that make a strong connection with the audience – where a high Cume Conversion Rate tells the greatest story of all.

In life, you run across different kinds of rock stars.

There are the obvious ones – extroverts, in front of the stage, commanding attention.

But there’s another kind of rock star. The one behind the scenes, the kind whose presence could be missed if you’re not looking closely enough.

Jennifer Donnelly and Mandy Couch are those kinds of rock stars for Coleman Insights. Their presence may not always be obvious, but their influence permeates throughout nearly every project our firm completes.

Jen and Mandy both carry the title “Senior Music Specialist.” You’ll discover in the notes from our recent lively and spirited roundtable discussion just how these two peas in a pod are massively integral to Coleman Insights’ success.

Coleman Insights’ Senior Music Specialists Jennifer Donnelly and Mandy Couch

It starts with their deep love of music from a young age. Jen says, “I would sit there with my tape recorder and record music off the radio. I would be so mad when my mom would come in and would mess up the recording because then I would wait for it to come on again.” Mandy talks about being raised on The Beatles, but remembers when Grunge broke as she was starting high school. “I was between Goth clothes and Grunge clothes. There was a station called Surf 107 in Wilmington, North Carolina, and that’s all I listened to.”

Jennifer’s dad co-founded The Make-A-Wish Foundation of Eastern North Carolina, which Jennifer was involved with after college graduation. Not long after, her unique music knowledge landed her a position at Coleman Insights (then known as Coleman Research) in 1994.

When Mandy graduated from college armed with an English degree, she was working at a record store when a customer referred her to Coleman Insights. After initially taking a front desk position at Coleman, Jennifer recognized Mandy’s passion for music, and she was promoted.

They have worked together as Coleman Insights’ “music team” for 23 years.

When you ask Jen and Mandy what their favorite part of the job is, you get a quick answer.

“Coding music.”

Every single song in the Coleman Insights music library has a sound code. How a song is coded has huge implications for studies because of the many unique features of a Coleman study, as Jen explains.

“When we’re coding songs, we aren’t just thinking about it for one thing. We’re thinking, how’s it going to cluster with other coded songs when we do their music test? We’re thinking about when we run a monitor analysis for a client (including showing the percentages of each sound code airing on the station). That must be correct. So we have to point out when one song doesn’t feel right. It affects everything else in our research.”

Sometimes the process includes spirited discussions about coding with each other, the Coleman Insights consultant team, and radio station programmers and consultants.

Once something is coded one way doesn’t mean it will always be coded that way, and that’s one of the advantages of having Jen and Mandy constantly evaluating the library. Jennifer uses Creed as an example of a band whose songs shifted in coding years ago. “When Creed first came on the scene, they were perceived as hard. Then, some songs crossed over to Pop. So, for example, when “With Arms Wide Open” crossed over, that made their other songs not seem as hard. We had to do some shifting of how we coded everything of theirs.”

Coding can change on certain songs, as occurred when Creed crossed over to Pop.

Jen and Mandy also need to consider how different formats see the same song differently. As Mandy explains, “A Hot AC station may see a song as AC, while a Mainstream AC may see it as Pop. You have to think about how it will fit on that specific station.”

These are just a few of the intricacies Jen and Mandy think about and deal with on a daily basis. Their work is behind the scenes and not always obvious, but it is highly worthy of our appreciation and respect.

Mandy makes a point about the client impact. “The way we code songs can make or break a station’s music presentation. Those codes go into music software, rules are put into place, and that’s the station’s sound to match the strategy. We’ve got to get it right.”