Following a recent post on the hidden secrets to creating insights, we got a load of questions asking for more watch-outs and tips for creating better insights. There are plenty that we’ve picked up over years of creating insights for some of the biggest companies in the world, and you can get all of it from our insights training programs (we only have capacity for one or two more programs this year, so please if you are interested, get in touch now). But here is one of the most important watch-outs when analyzing secondary data and creating insight. Check the origins of the data carefully.
Secondary data improves the chances of finding insights
One of the learnings I’ve shared in the past is to make sure that you are not just looking at the latest report you have. It’s one of the most common mistakes: we get a shiny new market research report and devour it. Then two weeks later its forgotten. The richness of insight often comes from blending together the findings from multiple reports. Grab consumer and shopper data. Grab quantitative and qualitative reports. Grab secondary data from other categories and search the web to see what might lie there. When we are searching for evidence (the second step of the insight process) more is almost always better.
But when we go off looking at secondary data, we do need to take care to understand its provenance. And in doing this there are therefore a number of key checks that we need to consider when handling secondary data.
Data might not lie, but the stories we tell with it?
Maybe!Research is usually done for a purpose, and then presented to show a set of results against that purpose. In its most benign form, the final report may be an edited version of the full report, there may be more to learn beyond that initial point. Sometimes however there is an attempt to deliberately use data to tell a particular story in a way that distorts that data. Rarely is the data faked, but perhaps what we do with the data might be misleading if taken out of context. Data rarely lies, but sometimes the stories we tell with it might!
A recent example was some data I saw presented on Influencer Marketing. It was presented as part of a marketing plan, and used to justify increasing spend on influencers. The marketer in question said that 70% of respondents said that they were more influenced by social media influencers than their friends. Sounds impressive right? But wait a minute. Where did this come from? Who were the respondents? Before we use secondary data, check its provenance!
Secondary data – Check who commissioned the research
First warning sign: the research was commissioned by a company that provided influencer marketing services. There is nothing wrong with that per se, but we always need to be sure that the research wasn’t self-serving. In this case, the research looked solid, but there was a little ‘misdirection’ going on. A bit of digging and it was clear that to be a respondent you had to have a social media account and you had to follow influencers. So, the true soundbite should have been ‘70% of people who follow influencers are influenced by them’. Not quite as sexy as the original quote, but now a little but closer to the truth.
Similar things can happen with all sorts of secondary data (sometime by mistake and sometimes by design – Don’t get me started on mobile phone use while shopping!) This blog isn’t about ethics, it is about being careful. Always go back and check the provenance of secondary data. Check who commissioned the research. Check the date, the sample, the purpose. Never look at someone else’s conclusions and assume that they are the full story, or even accurate.
Secondary data check: Is it out of date?
It is tempting, in these fast moving times, to assume that any research not completed in the last few months is probably out of date, but that simply isn’t true. While a lot is changing, and changing fast, that doesn’t mean that everything has changed. We do need to be careful, however. Things do change. Sense-check any secondary data source against the trends we do know. For example, if channel share has changed dramatically, then data about channel usage by shoppers might be out of date. If channel share hasn’t moved much, then that data is likely to be safer. Ask: Is it likely to be broadly accurate? If so it can be used, but used with care. Always date every data source as you build your analysis, so you never forget when the data came from.
Secondary data check: What was the original question?If you are looking at a research report and looking at a PowerPoint slide, be careful. The slide isn’t the data. The slide is an interpretation of the data by whoever created the presentation, and they had a different goal to you (more about this later). Always go back and understand what specifically was asked. I can’t emphasize this enough: so many times we’ve almost misunderstood what secondary data meant, because we assumed what the original question was.
Secondary data check: Check the questionnaire
While we are on the topic of questionnaires, it is always a good idea to go back and check this, rather than just looking at the report. Not all of the questions answered make it into the report, so there may be a lot of useful data that isn’t actually in the final report.
Secondary data check: Check the original data
Beyond the fact that there may be other stories buried in the data, there is another reason for going back to the original secondary data. Have you ever made a mistake in a calculation? Yes, me too! And whoever created the report you are citing, they make mistakes too. Go back and check the original data!
Secondary data check: Check the context
No research takes place in a vacuum. There is ‘stuff’ going on all of the time which may have impacted respondents. Consider all of the factors which might have had a significant impact on the responses you received. For example what marketing campaigns were running at the time? If you are looking at shopper research, what was actually going on in the store? A statistic such as ‘45% of people bought from a promotional gondola’ is of limited value unless we know how many promotional gondolas there were, in how many stores, etc. at the time the research was conducted. This is particularly prevalent on the area of shopper research – for more on the biggest mistake in shopper research, check this out.
To make secondary research high value, we need to curate it with care
Which brings me to a final, tangential learning. When you do commission primary research, do make sure you get, and keep, all of the documentation. Make sure you don’t make the most common mistake in shopper research. Get the PowerPoint presentation, but get Excel tables and the original database too. Keep the questionnaire, and any other contextual data too (e.g. how the research was set up, when it took place, what other conditions were important).
Secondary data is a critical input to the insight process, but without due care and attention, it is easy to go astray. With care and diligence (and effective research curation too) secondary research can play a massive part in any insight journey.