With the summer holidays approaching it’s always a good time to think about checklists – sun cream, guidebook, sunglasses, flip flops. Of course, at this time of year, we at MetaMetrics like to think about checklists for updating econometric models. Just as September is back to school, so it is also time when many businesses start their annual planning in earnest.
There’s often an accepted view that models simply need to be updated on a regular basis; quarterly, twice a year, annually, we’ve even been asked about monthly updates. Whilst naturally there is a good business to be made from routinely updating models on a regular basis, we often challenge this assumption and prefer to work towards a more needs-based approach. This has led us to develop a checklist that we use to decide when to update a model. Handily it also follows a neat ABCD mnemonic!
- Activity: It is essential to time your econometric study to best capture the activities you are interested in. We spend a lot of time working with clients to agree when that should be. Because econometrics is backwards looking, we need to choose that ideal moment to “cut” the data. Of course, there is never a perfect time: too early and we miss certain activities; too late and we don’t get results in time to inform planning. However, what we do know is that a rigid set schedule is hardly ever ideal. Some clients want calendar year results but that may end up cutting off the study halfway through a key activity or even a seasonal peak.
- Budgeting: One of the primary objectives of econometrics is to inform forward planning and budgeting. Econometrics can help in two ways: firstly, by optimizing the mix between channels at a certain agreed spend level and, secondly, informing whether the budget level itself is correct or whether it should be increased or decreased and what the resulting upside/risk is to business performance. As a result, it is important to bear in mind whether the delivery of results aligns with your planning process. There are often 2 stages to this: when are overall budgets set and then (usually later) how are they allocated by channel? Econometrics can inform both decisions.
- Creative: Econometrics is good at determining whether a creative change has been effective in increasing ROI. It is possible (with care) to strip out all the other effects such as cost, deployment, and timing to isolate the effect of the creative itself. For some clients we deploy our unique RapidROI product, specifically to determine the in-market impact of a change in creative. This can be done at any time as a supplement to the main econometric model and is more agile, by which I mean faster and cheaper! By doing this it doesn’t derail the timing of the main study.
- Decision making: Decisions need to be made at different times. If we are looking to influence TV spend levels or weekly flighting, this has to be done well in advance to take account of Advance Booking deadlines. As a result, if we are using econometrics to inform that decision, we need to plan around these “AB” deadlines. However most digital channels can be changed at much shorter notice and if this is the focus of the econometrics, then the study can be delayed to run much later. So, the ideal approach is to think ahead to what actual decisions you are looking to affect and the lead times around those.
Hopefully, this has given some food for thought as to the various considerations around the timing of an econometric modelling study. As ever, if you have any questions please do get in touch, at any time!