Coleman Insights recently introduced the FACT360 Strategic Music Test. FACT360 is online music testing done right through the latest sampling techniques and data collection capabilities, and includes the same benefits that Coleman Insights has provided through its FACT Strategic Music Tests for more than 20 years. These benefits help radio stations build the most appealing and strategically on-target libraries possible.
In the spirit of our FACT360 launch, we’d like to present the second in a series of five blogs authored by Coleman Insights executives covering important considerations about music testing and music strategy. This blog is written by vice president Sam Milkman and addresses how to assemble the right sample for your next library test.
There is an old saying about market research, “Garbage in, garbage out.” It means that a study that is not designed correctly—including, among other things, employing a sample that does not represent the population you are trying to measure—will produce results that could lead to the wrong conclusions. One thing I learned when working in programming at Z100/New York, K-Rock/New York and WMMR/Philadelphia that has been reaffirmed since joining Coleman Insights is how vitally important it is to make sure the “right” people participate in your station’s library test. The specifics of who should be in your next test are highly customized for each station, but I will share a few basic rules of thumb that any station conducting library testing should follow.
First, you should test your music with listeners who are randomly recruited to be in your study. In other words, your sample should not be culled from your station’s listener database or from people who respond to a banner on your website inviting them to provide feedback on the music you play. Sure, this approach reduces the cost of acquiring respondents considerably and could have marketing value for your station; unfortunately, it may result in a sample that poorly represents your target audience. Researchers call this a “self-selecting sample,” and anyone who has taken a market research course quickly learns that such an approach is a definite no-no.
The other problem with music testing samples that are built this way is that the respondent knows the station for which the music test is being conducted. This invalidates any effort to objectively measure the perceptions respondents have of your station.
A second major tenet of library music testing sample design is that it should be driven by your strategic research. Strategic studies like Coleman Insights’ Plan DeveloperSM and FLIPSM studies should provide a station with a clear vision of its target audience in terms of age, gender, ethnicity, music tastes and listening habits. These studies should also give a station insights into how strong its position is and how deeply it has penetrated its market’s appetite for what it offers. Those two factors are crucial because library testing for strongly-positioned stations should generally employ samples that are relatively focused on the station’s existing listeners; stations with weak positions that are not deeply penetrating their market’s appetite for the music they play should generally take a broader approach.
Your strategic research should identify where the “true” demand for your music strategy exists. In general, the age range should rarely cover more than 15 years, for most formats the focus should be on one gender over another and—unless a station is very early in the development of its music position—most if not all of the respondents should be existing users of your station.
For our FACT360 Strategic Music Tests, we usually advocate our TSL MaxSM sampling approach for stations with well-developed music positions. TSL Max involves recruiting an appropriate mix of your station’s heavy, medium and light listeners to gain insights into the music that caters to the tastes of your heaviest TSL-generating core listeners versus the music that helps keep your station’s Cume appeal as broad as possible.
Third, and perhaps most important, ensure that you uphold your quotas. Once you determine the “right” sample parameters for your library test, you must stick with them. If you have determined—for example—that 50% of your sample should be 24- to 30-year-olds and the other half should be 31- to 37-year-olds, do not deviate from that if you are having more success recruiting one age cell over the other. If 24-37 truly represents the age target for your station, you should look to that age group to provide you with the data you need to make the right decisions about what songs to play and how often to play them.
Even though I have only scratched the surface of all of the considerations here, sampling for your library test is not rocket science. It does, however, require a scientific and disciplined approach and I hope these rules provide you with some guidance to successfully test your station’s music library.
