Here at MetaMetrics, we’ve just built our first Covid-proof marketing mix model for a client to see how effective their media spend has been during the pandemic. Here’s what we found.
As the worst effects from Covid-19 hopefully recede, it’s time once again for econometricians to step up and tell marketers, with the benefit of hindsight, what you should have done. Therefore, we’ve recently built our first “Corona model”; that is, a model of data that may (or may not) have been done over by Covid-19. The ‘may or may not’ is crucial so keep that in mind as we move on.
It wasn’t the first set of Covid-infected data we had received from a client. We’d already seen a bunch of series that looked like this. (Spot the pandemic outbreak, anyone?)
In these instances, we did the only sensible thing and chopped off the last data point from our models (with, of course, our clients’ agreement). After all, we could all agree this was an anomaly caused by the outbreak and that’s what you do with this type of data.
But this ‘anomaly’ carried on. It became the new ‘business as usual’ and so it was crucial to understand if media spend was working in this new norm. Therefore, we found ourselves looking at sales data that ran right across lockdown and being asked to measure the effect of the TV campaign alone, that is, the impact of the TV less any minor perturbances arising from, say, a global pandemic.
“Can it be done?” our client asked us. They weren’t the only ones asking.
Turns out it can.
What did we do?
What we did was explore a range of possible proxies for Covid-19.
Backtracking a little, something like a Covid outbreak undoubtedly has the potential to materially affect sales. Some, perhaps selling services via the internet, will do well from Covid. Many will fare worse. And, generally speaking, in an econometric model we should test anything that has the potential to affect sales.
The problem is that marketing mix models work off weekly time series data, i.e. a bunch of numbers, which “Covid” isn’t. The challenge with things like Covid is to find suitable, numeric proxies, suitable for testing and, if proven, including them in our model.
So, we looked for proxies – metrics that we believed would vary with the impact of Covid. That presents another challenge – the sheer amount of data available around Covid. There are infections, admissions, deaths (including or excluding care homes), Rs, and a whole host of others besides. (For a discussion of the limitations of these, see any newspaper.)
In the event, after exploring a range of different metrics for our client, we found that what fitted the data best was a Google Trends index for Coronavirus-related searches in the market we were modelling. That might seem an odd choice – why use a soft proxy rather than a hard measure, such as reported infection rate? But actually, the impact of Covid on our behaviour is far more wide-spread than just to those who presented as infected from the virus. And it turned out that the volume of people researching Coronavirus had a close relationship with sales of our client’s products.
Here’s the Google Trends Index for the UK. Notice how the volume of people Googling ‘coronavirus’ surges in the weeks prior to lockdown, and then ebbs away as lockdown persists. And, sure enough, sales dipped around lockdown, and then gradually recovered as people got used to the new arrangements.
From this, it was only a short step to being able to extract a robust measurement of the TV effect, without the Covid impact, from the data and calculate the ROI in the usual way. What our modelling showed, for this client, was that their media ROI over the lockdown was consistent with measurements of previous, pre-Covid campaigns.
What lessons can we learn from this? We’re loathe to generalise from the one case, but in this instance, there was enough consistency in people’s behaviour before and during Covid to build stable econometric models. The world hasn’t changed all that much (for this client) and the new normal looks rather like the old.
What we can say is that, with some experimentation and patience, it IS possible to build robust models that take into account whatever effects Covid has had on your business, whilst still producing reliable media and marketing measurements.
Here are our top 3 tips for doing so:
1. Don’t ignore it… but don’t let it defeat you – it can be done
Start with what you know and build out from there. Put in all your activity and see how the model fits reality and what the Covid shaped gap between your model and reality looks like.
2. Find lots of behavioural proxies and test them
It might be footfall, or economic data, or a different search term relevant to your category, it costs nothing to test these.
Consider: does it need to be lagged or decayed, or even accumulated (see chart, below).
3. Above all, get buy-in for your approach
In the world of proxies, there are no hard-and-fast rules. Make sure you create a consensus around your approach. What you are using may be counter intuitive at first glance to management and need explaining.
For further help on calculating your media spend ROI, at any time not just during a global pandemic, do get in touch. We’re happy to have a chat over a virtual coffee with you to discuss your situation and see if we can help