CTV and OTT measurement deep dive
In part 1 of our over-the-top (OTT) blog series, we shed light on the advanced TV landscape by defining terminology, introducing AppsFlyer’s OTT measurement integrations, and describing key trends to watch in the OTT market.
For performance advertisers, defining your attribution methodology should be the first step to determine your OTT test strategy. That’s why we’re using this post to dive deep into the topic of advanced TV measurement with a technical focus on OTT and connected TV (CTV) attribution.
TV measurement overview
Advertisers have long accepted the limitations of traditional TV measurement.
Unlike digital channels that rely on URLs, pixels, postbacks and S2S/SDK integrations to provide one-to-one, deterministic measurement between ad engagements and conversions, linear TV relies on probabilistic attribution.
TV advertisers receive standard reporting on estimated impression delivery by demographic (e.g. 18-34 age group) based on TV ratings and Nielsen panel data, but if they want to understand the impact of those impressions they’ll have to invest in separate research. This is typically done via incremental lift analyses or paid brand studies to estimate the likely effect of TV on correlated metrics like awareness, acquisition and sales.
AppsFlyer uses a similar method to attribute mobile installs to TV ads via time – and location-based signals at no added cost.
Enter CTV.
While internet connectivity has the potential to provide a more accurate link between TV ads and cross-device actions in the same household, the development of true people-based attribution is still nascent. AppsFlyer is proud to offer deterministic attribution of OTT app downloads occurring on the same platform as the CTV ad, but for now, attribution from CTV ads to other devices relies on the same probabilistic methodology as traditional TV.
Some marketing cloud companies have started to offer “identity graphs” which use household- and user-based signals to provide advanced cross-device attribution from CTV, but these offerings are not yet integrated with any major MMP.
Deterministic measurement: SDK/S2S integration
There are currently two reliable options for holistic attribution of OTT app downloads occurring on the same OTT platform as the CTV ad:
- Direct integration with each platform provider for data pass-back (a manual and cumbersome process for those advertising across platforms)
- Working with a centralized attribution partner (e.g. AppsFlyer)
Integration with AppsFlyer allows advertisers to see all their OTT installs in one place, rather than aggregating separate reports from each OTT platform provider. In addition to streamlining the analytics process, AppsFlyer also provides advanced measurement options.
Setup unlimited in-app events, configure custom attribution windows and seamlessly integrate OTT media sources to facilitate deeper insights and standardization across OTT platforms.
We currently offer the following direct integrations, with more platforms coming soon:
| OTT Type | Platform | SDK | S2S | Notes |
| Streaming Box/Stick | Fire TV | X | X | SDK requires out of store APK |
| Roku | X | X |
SDK requires out of store APK; | |
| Apple TV | X | X | N/A | |
| Gaming Console | Xbox | X | X | N/A |
| Smart TV | Roku TV | X | In development | |
| Samsung | X | |||
| LG | X | |||
| Android TV | X | Android TV includes Nvidia, Philips, Sharp, TCL, Toshiba and Skyworth TV brands |
Once your OTT app is integrated with the AppsFlyer SDK, it takes just a few clicks to enable paid attribution for integrated media sources. While optional, server-to-server (S2S) integrations allow you to send AppsFlyer events back to each OTT partner and media source for optimization purposes.
Probabilistic Measurement: Lift tests & device graphs
Deterministic measurement will always be more accurate than probabilistic methods, but for some advertisers it may be worthwhile to invest in additional analytics that can give them a broader picture of the impact from CTV campaigns.
This is not only helpful for OTT app owners, but also for mobile app owners who wish to use CTV advertising to generate awareness. For the mobile performance advertisers in particular, running CTV campaigns without any idea of their impact on product consideration or app installs may seem like a waste of money.
Luckily, some media partners and marketing analytics companies are starting to offer solutions that help answer this plea. There are two main ways to do this:
- Conversion lift test
- Cross-device graph
Conversion lift test
Conversion lift tests can only be executed by a DSP who is running media across multiple devices, and requires turning off other media to avoid test contamination.
Prior to launch, the DSP divides your target audience into an exposed group and a control group.
The test group sees your ads, while the control group sees placebo ads.
After the campaign ends, the advertiser analyzes the install conversion rate among between the test and exposed groups to understand the likely impact of ad exposure. This can then be used to benchmark eCPIs from CTV to mobile.
Unfortunately, these tests come at a hefty price as the advertiser must pay for ads delivered to the control audience.
A similar, potentially cheaper approach to this problem involves ghost ads.
With this method, you only pay for your own ads. On the backend, the DSP tries to randomize a select audience exposed to ads other than your own – essentially replicating the control group. Ghost ad tests are either offered as added value with a significant minimum spend (e.g. $200K), or come at a flat fee that should cost less than the placebo ad approach.
It should be noted that both of these approaches are based on correlation analysis that may be impacted by selection bias. This may lead to different results between tests, requiring iterative testing to come up with accurate predictive models.
Get the latest marketing news and expert insights delivered to your inbox
Cross-device tests
Cross-device graphs are arguably the most comprehensive approach to date for cross-device attribution from CTV.
The above graphic illustrates a scenario where the advertiser is using a data management or marketing cloud platform such as Drawbridge, Tru Optik, Samba TV, Data + Math, Adobe, or Oracle to aggregate data. IP is typically the main connecting metric for this approach, while other metrics such as device ID, user agent, time and geo are used to validate the accuracy of cross-device graph connections.
The end result is a multi-touch attribution report that estimates which app installs were driven by media across devices, i.e. CTV to mobile and vice versa.
The main drawbacks to this approach are cost and accuracy.
Involving a custom analytics partner can cost upwards of $10K/month. The alternative to this is finding a DSP or programmatic partner who can offer cross-device attribution as added value (e.g. The Trade Desk), but each partner has its own approach with its own limitations and minimum spend requirements.
Cost aside, cross-device graphs may lend more accurate results than conversion lift tests, but they still fall short of deterministic methods. Any associations from cross-device graphs should be taken with a grain of salt.
Looking forward
Overall, CTV measurement is still extremely fragmented, not unlike the wild west of mobile attribution a decade ago. However, the high growth rate of CTV will naturally lend itself to technical advancement over time.
Moreover, the synergies between OTT apps and mobile present clear opportunities for app marketers to start testing OTT television sooner than later. In our next post, we’ll share more details on our integration with Amazon Fire TV, featuring advertising opportunities and measurement best practices.
