How Marketing Mix Modelling Works

The depth of Ullswater at Pooley Bridge varies between around 1m and 3m and if you squint at this chart of its levels since 2021, doesn't it look a bit like the sales line for a brand that's running lots of promotions?

We're going to create a model to explain what's changing the depth of the water, in exactly the same way as an econometrician would build a model to explain what's driving the sales of a brand.

A brand's sales are driven by the four P's - product, price, place and promotion. Water in a lake is driven by just one P - precipitation. For this model, rain is our advertising. More rain is more water in the lake is more sales.

You'll see in a moment that even though there's only one driver in our model, getting it to work properly is actually surprisingly tricky.

When our model works properly, it's able to explain why the water level has changed in the past. The blue line is the actual depth of the water and the red line is a sketch of roughly what we hope our model's shape might look like. If movements in the red model line mostly fit the shapes in the blue actual line (and you didn't cheat) then you've got a decent model that explains what's happened in the past. Some statisticians reading this are recoiling in horror but let's just say that 'and you didn't cheat' can get quite complicated and move on.

Before we get to measuring rainfall, what if we tried something simpler?

Water levels in the lake are generally higher in winter and lower in summer. What happens if we make a model and don't tell it anything about rainfall at all? We only give the model some basic seasonality data that says sometimes it's winter (October to March) and sometimes it's not winter and then we'll see how well it can explain the water's depth.

In technical language this is called a dummy variable. Our seasonality dummy variable has a value of 1 when it's winter and 0 all of the rest of the time.

It works! Sort of. The red line is the model's understanding of the water level and we can see that it's higher during winter. Ideally you want the red model line to track all of the changes in the blue actual line but this very limited model has only captured some basic shapes.

The model hasn't explained any of those big spikes, only that the water level is generally higher in winter and lower in summer.

It's important to think about what we just did because we made a choice. We modelled that the water level is higher in winter 'just because'. Not because it's caused by anything that we've explained, it's higher in winter just because it is.

This fairly silly model is even potentially useful in a limited way. If you were an alien who knew nothing about lakes, now you'd know - on average - how much more water they have in them during winter.

Analysts make choices like this all the time and that's fine. It's part of the job, but it also illustrates that an econometric model isn't at all a standardised thing.

Who you commission to build your models is extremely important, as one analyst might measure that winter seasonality is causing the water depth to rise (and we should acknowledge that they wouldn't be strictly wrong in that statement) but another analyst - a better, more diligent analyst - would take the next step and use rainfall data instead of seasonality to make a much more useful model, like we will now.

Of course we already know that rainfall will end up in the lake and if we grab the data and chart it then that's exactly what we see.

The depth of the water spikes upwards during periods of rain.

We can use this rainfall data to build a better model. We're still trying to understand changes in the daily water depth, but now instead of a seasonality variable to explain why it's changing, we give the model daily rainfall data.

Is our new model any good?

Well... It's different. You can see it's trying to track the actual water depth but it's not working properly yet.

Marketing mix models compare the changes in one variable with the changes in another. If they're 'positively correlated' meaning they rise and fall together, or they consistently oppose each other - one always goes up when the other goes down - then the model will use them.

So what happened to our model? It looks a bit weird.

The model has tried to fit the spikes in water depth but hasn't done a very good job of it and its baseline is much too high. When the water level is lower in summer, the model doesn't understand that at all.

It's obvious to us that the depth of water rises when it rains so why does our new model not really work?

It's tough to see the problem when you're looking at three whole years of daily data. We need to focus on one of the problem periods.

For now, we need to do something to those rainfall numbers to make them look like the shapes that they create in the water depth. I said earlier that building an econometric model doesn't use much maths day-to-day and it doesn't - the experience is much more like doing a jigsaw puzzle. We're always looking for the next shape that we need and at the moment our rain data is much more spiky than our lake depth data.

We want a shape that builds up gradually when it rains and then takes some time to die back down again.

We do that with a decay rate. We make a new variable (an input) for our model, where the value of 'rainfall' isn't just that day's rainfall, it's that day's rain plus some carried-over amount from the past. It might help to think about our new variable as a leaky bucket - you put water in and it slowly leaks out again, so it's always got today's rain in it but also some of yesterday's and the days before that...

Your next question might be to ask how we know the amount of rain to retain from the past. What decay rate do we use?

Our analyst is making choices again. Guided by their experience and by the behaviour of the model they're creating, they look for a value that fits the data. Going back to our leaky bucket analogy, they try buckets with big holes and buckets with little holes and they see what fits.

When we're modelling water in a lake this is fairly easy because only rain is affecting the depth of the water and the impact of rain is big, so you can quickly see whether you've got it right or not.

When we're building a marketing mix model it's not easy at all. There's loads going on - competitor activity, price changes, promotions and multiple advertising channels, plus the uplifts from advertising are usually smaller than the spikes from promotional discount offers and so are harder to see. Analysts lean a great deal on their experience, on benchmarks and on continually iterating and improving their model to get to their best estimate of the truth. Again, who builds your model is extremely important because statistics alone cannot do these things for you.

It turns out there isn't one decay rate that will make the model work. We need two of them.

The shape that rain creates is an immediate spike in the depth of the water and a slower rise and fall that endures over a longer period of time.

We often see a similar shape when we look at brand search data while TV adverts are running. On the day that you have a big TV spot, searches are much higher but there is also a more subtle, sustained rise in searches throughout the length of the campaign and extending on after that campaign has ended.

Once we give the model these two different shapes to work with, suddenly it can explain the depth of the water.

It's not perfect - if you look carefully then you'll be able to spot times when the red model and blue actual lines don't match - but it's pretty good. We could stop here and be happy that we'd built a reasonable picture of how rainfall affects the depth of Ullswater.

Why does it sometimes not fit? There are all sorts of effects we might not have captured. Temperature is probably important because rain and snow behave differently, and we're using local rainfall data but maybe sometimes its only raining further upstream and that water still ends up in the lake. Models are never perfect, we work on them until they're capable of the job that we need them to do.

Now that we have our model, what do we do with it? We'll want to ask some 'what if?' questions and see what might have happened under different scenarios.

What would be the result of an extended period of rain? We can invent some new rainfall data, put it back into our model and ask for a prediction.

Let's pretend that July 2023 was a complete wash-out. We'll invent our own rainfall data that says it rained every day for the entire month and ruined everybody's summer holidays. What would have happened to Ullswater?

As you'd expect, the model predicts that the water level would have been much higher. Marketing mix modelling is good at questions like this, where we play with scenarios that don't extend too far outside the scope of the data that's been used to create the models in the first place.

In advertising, we can play with campaign budgets and different combinations of media channels to get predictions for what would happen to sales. For example we could produce a prediction for what might happen to July's sales if we run bigger promotions and more advertising than last year. As long as we don't go too far outside of the campaign budgets that the model has seen before, we should get sensible predictions.

What would have happened if July 2023 had no rain at all?

Our model can generate a new prediction that the water level would drop throughout July, hitting a level below 1 metre before it started to rain again in August. There would still be a lower water level for quite some time afterwards, due to the decay rates we saw earlier. A lack of rain now will have an impact into the future.

When we're modelling advertising rather than rain, this is great news. We can measure the benefit of advertising now and measure the benefits that it creates into the future.

If we were only trying to predict what happens for the next few months, we'd be in a great place. We've got a model that explains the past and we can use it to make predictions about what will happen if we change its inputs. That's very useful.

But we want to measure ROI. All of the benefits created by advertising. And that's a problem.