Home On TV & Video How The YouTube Scandal Exposes A Double Measurement Failure

How The YouTube Scandal Exposes A Double Measurement Failure

SHARE:
Nico Neumann, assistant professor and fellow, Centre for Business Analytics at Melbourne Business School

By now, most people working in advertising have heard about the recent YouTube scandal

A comprehensive report from measurement company Adalytics provides a long list of cases where TrueView ads didn’t seem to adhere to Google’s quality standards. Ads were served on third-party sites and apps via the YouTube partner network, known as GVP. 

Many advertisers were unaware their TrueView ads were actually running on GVP sites. Of particular concern is that numerous ads placed on GVP were found to be muted and/or auto-playing, sometimes without any accompanying video content.

Google and other traffic verification companies have released several press statements pertaining to the scandal. But discrepancies across the various attempts to quantify the issue and address everyone’s concerns have just raised more questions.

The statistics war

While everyone largely agrees that problematic or inferior GVP ads are not what media buyers expect when paying premium dollars for TrueView placements, opinions diverge on the scale of the uncovered problem. How many campaigns actually include GVP inventory? And how many GVP TrueView placements may end up being outstream, muted, auto-played or hidden?

The Adalytics report found between 50%-90% of the TrueView billable cost of audited campaigns ended up on GVP. But “Pixability found that across all their video campaigns only 11% of their ads ended up on GVP, whereas IAS found that GVP accounted for 21% of campaigns with both YouTube and GVP inventory,” according to Google’s official statement.

This range of numbers has made people wonder who to believe. However, such a debate completely misses the point. 

The various reported percentages represent samples from diverse campaigns and varying time periods. They appear to focus on different units of measure (billable views vs. impressions). And this matters a lot for a product like TrueView, where only a fraction of impressions results in a completed, billable view.

Therefore, it’s impossible to draw any absolutes from these summary statistics. It’s like measuring the temperature in Celsius and Fahrenheit at different months of the year in different countries at different times of the day – and then asking which value best describes what the weather’s like on Earth. 

Interestingly, Google – the only company that should know the exact numbers – just mentioned in their press release that “the overwhelming majority of video ad campaigns serve on YouTube, not GVP.”

Subscribe

AdExchanger Daily

Get our editors’ roundup delivered to your inbox every weekday.

Exposing a double measurement failure

The most important lesson we can draw is that, too often, there is a double measurement failure.

The first failure relates to the walled garden mentality. Currently, no verification vendor is allowed to measure campaigns on YouTube. Instead, advertisers must rely on the information that Google provides through Ads Data Hub for their assessment of campaign quality. This basically means it’s impossible to determine how many GVP campaigns suffer from the reported problems.

Let’s be clear about this concern. Imagine you are a food safety and health inspector, but are never allowed to enter the kitchen of an establishment you are supposed to examine. Instead, the restaurant only sends you some sample photos. How many restaurants would send a picture showing a dirty kitchen or health code violation?

Yet, any ad-placement-level analysis only represents the first measurement layer in the media procurement chain. And this step will always depend on the goodwill of the publisher or platform to allow any external JavaScript tags or tracking pixels. Even in a world with privacy regulations increasing, there must be a way to allow external parties to properly run audits while protecting sensitive information.

A second layer of measurement

Even without access to ad placement data, there should always be a second layer of measurement. A working ad effectiveness method should be in place to help identify any campaign inefficiency, be that viewability, fake traffic, or poor ad placements. This should serve as a safety net in case the first layer fails.

Let’s not forget – unseen ads should not have an impact on any meaningful marketing outcome, be it brand awareness or sales.

So the question is: How many marketers have a method in place that would flag the performance of a problematic GVP campaign? Too often, marketers still prefer volume-related vanity metrics or benchmark-inflating attribution models that don’t reveal campaigns without incremental advertising effects.

What actions can marketers take today?

There are four questions that any brand should ask their agency or media buyer now:

  1. How much are our campaigns affected? Conduct an independent review of your video placements using log-level data and digital forensics. Don’t rely on summary reports.
  2. How much of YouTube’s inventory is on GVP? How much is outstream versus instream? There should be no excuse to hide such basic measurable information.
  3. How can we minimize exposure to the problematic GVP inventory? Untick the box when running campaigns and avoid performance choices that do not allow removing GVP inventory if you want your money to be spent on actual YouTube ads.
  4. Would your measurement methods pick up any problematic GVP or other video campaigns? It’s never too late to review your media analysis choices and opt for more robust scientific methods, such as ongoing experimentation.

Let’s not waste time arguing about industry statistics. They are all averages. What matters is how your brand is affected.

On TV & Video” is a column exploring opportunities and challenges in advanced TV and video. 

Follow Nico Neumann and AdExchanger on LinkedIn.

For more articles featuring Nico Neumann, click here.

Must Read

Google filed a motion to exclude the testimony of any government witnesses who aren’t economists or antitrust experts during the upcoming ad tech antitrust trial starting on September 9.

Google Is Fighting To Keep Ad Tech Execs Off the Stand In Its Upcoming Antitrust Trial

Google doesn’t want AppNexus founder Brian O’Kelley – you know, the godfather of programmatic – to testify during its ad tech antitrust trial starting on September 9.

How HUMAN Uncovered A Scam Serving 2.5 Billion Ads Per Day To Piracy Sites

Publishers trafficking in pirated movies, TV shows and games sold programmatic ads alongside this stolen content, while using domain cloaking to obscure the “cashout sites” where the ads actually ran.

In 2019, Google moved to a first-price auction and also ceded its last look advantage in AdX, in part because it had to. Most exchanges had already moved to first price.

Thanks To The DOJ, We Now Know What Google Really Thought About Header Bidding

Starting last week and into this week, hundreds of court-filed documents have been unsealed in the lead-up to the Google ad tech antitrust trial – and it’s a bonanza.

Privacy! Commerce! Connected TV! Read all about it. Subscribe to AdExchanger Newsletters

Will Alternative TV Currencies Ever Be More Than A Nielsen Add-On?

Ever since Nielsen was dinged for undercounting TV viewers during the pandemic, its competitors have been fighting to convince buyers and sellers alike to adopt them as alternatives. And yet, some industry insiders argue that alt currencies weren’t ever meant to supplant Nielsen.

A comic depicting people in suits setting money on fire as a reference to incrementality: as in, don't set your money on fire!

How Incrementality Tests Helped Newton Baby Ditch Branded Search

In the past year, Baby product and mattress brand Newton Baby has put all its media channels through a new testing regime for incrementality. It was a revelatory experience.

Colgate-Palmolive redesigned all of its consumer-facing sites and apps to serve as information hubs about its brands and make it easier to collect email addresses and other opted-in user data.

Colgate-Palmolive’s First-Party Data Strategy Is A Study In Quality Over Quantity

Colgate-Palmolive redesigned all of its consumer-facing sites and apps to make it easier to collect opted-in first-party user data.