Our two-part, warts-and-all guide to what econometric modelling is, what it isn’t and how and when to use it, in 10 steps
This is Part 1.
1. What’s it called? Econometrics, Mix Modelling, Market Mix Modelling, Econometric modelling. It’s all the same thing.
We try to avoid calling it “attribution.” Although it does indeed attribute sales to sales drivers, that word is generally used to mean something different in the context of digital measurement and, frankly, just confuses everybody.
Econometrics is typically ‘done’ on sales data and can be done to quantify the effects of media, pricing, promotions or everything. The latter is often what’s called Market Mix Modelling. Note that, even if your questions are purely media-related, you should still control for the other major drivers – pricing, promotions, distribution and so on – as these tend to be large and omitting them from the model will mess up your media results.
Confusing digital attribution with econometrics is a bit like doing this
2. What’s it for?
It’s a part of the decision-making process, one input amongst many, to be taken together with other research and business knowledge and judgement. (Assimilating and distilling this into actionable recommendations is what insight professionals get paid for.)
What it ISN’T is an all-knowing computer model that will tell you what to do with accuracy and certainty. Sorry.
There are often two opposing groups in terms of their attitudes to modelling. Both are equally dangerous. No, in fact the first is more dangerous. These are people who invest too much importance in the model and become slavishly reliant on it, wanting it to explain and predict everything.
The second group are those who believe it’s all hocus pocus, statistical nonsense. “Our business can’t be modelled because our category or brand is ‘different’. We act on instinct, we know what’s right.” We hear this a lot. In some cases, it’s true – some brands and situations are unmodellable. It’s a good deal less true one might think, however, and usually there is some insight to be gained from modelling that complements gut-feel, either supporting our instincts or, occasionally, reining them in.
Not a silver bullet
3. What is it?
It’s a statistical process that uses maths to quantify the impact of activities (advertising, pricing, availability, seasonality etc) on the thing you want to measure (usually sales). This is usually done via a process called multiple regression. It then applies the same statistical tests you see in other research (e.g. 95% confidence) to see whether we can be confident that that driver has a quantifiable impact on sales.
Some maths – pure, not statistics, but who’s going to know?
It’s data agnostic. The model does not know or care what drivers you put in. It doesn’t know that the business invested a lot of time and effort in a certain activity and is desperate for it to succeed. Or what Someone’s pet theory is (or, more to the point, who that Someone is), nor whether the ad moved people to tears, or even won an IPA award. It is a dispassionate look at cause and effect: what happened and what impact did it have?
4. What does it do for me?
Econometric modelling quantifies the incremental effect of activities at a relatively high level.
What does that mean? Incrementality is key here. Say I drop the price of my product. I will sell more. But some of those sales will be to people who would have bought it anyway, so those sales would not be counted as incremental. The same principle applies to paid search. If I drive someone who was going to buy anyway to Google and they click on a sponsored link before they buy, that sale isn’t incremental either. You can’t calculate a true ROI unless you measure what’s truly incremental and only econometrics can do this.
Google. We’ve all done it, although maybe not quite like this. The question is – what would we have done otherwise?
What do we mean by “at a relatively high level”? Econometrics is good at quantifying broad effects of activities. What it probably won’t tell you is which of the 75 digital banner ads you ran performed better. It’s just too granular, small scale. Use other techniques such as click through rate to get a handle on this.
5. What it does well
Econometric modelling is great at giving you a single view of the effect of all the different potential drivers in one analysis. Very often pricing analysis is done by one team, media by another, digital by yet another. Weather by, well, nobody usually. Each team does its analysis in a different way and (shhh!) may have vested interests in the answer. Econometrics sweeps all this away and compares all activities on a level playing field. This is often why the most interested person is the Managing Director rather than the functional group heads; or, in the case of a media-only study, the Marketing Director wanting to compare offline and digital media, for instance.
See Part 2 for what econometrics does badly…
Tom Lloyd