Building MMM with an external agency is a common path. The relationship needs care though, or the model ends up serving someone else’s needs. I have been on both sides of this. Here is what I learned.
Why use an agency
Agencies bring:
- Specialized skills and tools
- Benchmarks across clients
- Extra bandwidth when your team is stretched
- Political cover for findings nobody wants to hear
Those are real reasons.
Red flags
Black box methodology
If the agency cannot explain the model spec, the coefficients, and how they validated it — walk away. You need to understand what is driving the results.
I once inherited a model nobody internally understood. When the numbers stopped matching reality, we had nothing to debug with. It was painful and slow.
Too much precision
MMM has wide confidence intervals. Always. If an agency tells you “TV drove exactly $4.2M in incremental revenue” with no uncertainty bounds, they are selling. Be skeptical.
Optimizing for the CMO’s mood
Some agencies tune the model until it shows what the client wants to see. Insist on pre-registered specs. If they push back hard, that tells you something.
What to ask for
- Full model documentation — functional forms, priors, variable transforms
- Holdout validation — out-of-sample accuracy, not just in-sample fit
- Sensitivity analysis across reasonable parameter ranges
- Raw output files, not just a polished deck
- Access to the code, if they will give it
How to make the relationship work
The best agency engagements feel like partnerships. You share business context. You explain why a particular result would be surprising. You push back when something does not make sense.
Your job is not to accept deliverables. It is to understand the model well enough to defend or critique what it implies.
The agencies I have had the best results with welcomed the pushback. The ones who got defensive about it — I did not work with them again.