Building Marketing Mix Models (MMM) with external agencies is a common path. But the relationship requires careful management to ensure the model actually serves your needs. I have been on both sides of this, and here is what I have learned.
Why Work with an Agency?
Agencies bring:
- Specialized expertise and tooling
- External benchmarks across clients
- Bandwidth when your team is stretched
- Political cover for controversial findings
Red Flags to Watch
1. Black Box Methodology
If an agency cannot explain their model specifications, coefficient estimates, and validation approach in detail, walk away. You need to understand what drives the results.
I once inherited an agency model that nobody internally understood. When the results stopped matching reality, we had no way to diagnose why.
2. Unrealistic Precision
MMM estimates have wide confidence intervals. Any agency promising “TV drove exactly $4.2M incremental revenue” without uncertainty bounds is overselling. Be skeptical.
3. Optimizing for Client Happiness
Some agencies will tune models until they show what the CMO wants to see. Insist on pre-registered specifications. If they resist, that tells you something.
What to Ask For
- Full model documentation including functional forms, priors, and variable transformations
- Holdout validation showing out-of-sample prediction accuracy
- Sensitivity analysis across reasonable parameter ranges
- Raw output files not just polished decks
- Code review access if possible
Building a Productive Relationship
The best agency engagements feel like partnerships. Share your business context. Explain why certain results would be surprising. Push back when something does not make sense.
Your job is not to accept deliverables. It is to understand the model well enough to defend or critique its implications.
The agencies I have had the best results with are the ones who welcomed my skepticism rather than getting defensive about it.