Ad Ops Tools

How to Catch & Kill Mobile App Store Redirect Ads

January 3, 2017

We’ve all seen them; you’re casually browsing your favorite app or playing a game on your phone and suddenly you’re being redirected to the app store to download Candy Crush. What gives? It’s another obnoxious mobile app store redirect ad that’s automatically sending you to the app store without a click.

These are the pop up ads of the mobile age and virtually everyone hates them, including the chain of mobile publishers, exchanges, and other ad tech that unwittingly served them in the first place. But they can be quite difficult to find; because no one wants to serve them they are purposefully targeted and served in a way that makes them difficult to replicate. TechCrunch wrote a good overview of the complexity of this problem in 2015, and also linked to Sergei Frankoff’s detailed technical description on the Sentrant blog of how frame-busting JS combined with 302 redirects was able to automatically open the app store.

Below is a step-by-step guide designed for ad operations teams to demonstrate how to use Charles Proxy to identify and eliminate a mobile app store redirect ad, though it unfortunately won’t act as a detection system or a blocker of any kind. Thankfully most exchanges have found ways to ban frame busting code like the one mentioned in the TechCrunch article at this point.

Step 1: Start recording your web traffic with Charles Proxy

Charles Proxy is a program that will sit between your browser and the internet and record all the different interactions when loading a web page, even those you can’t see in the source code. The benefit of using Charles is it doesn’t matter how fast the page redirects or how many parties are involved in the process. Charles will record everything and let you meticulously search through all the interactions at your own convenience. It has a free trial version, but if you work in ad operations you should just buy a license. The tool is essential for all sort of debugging needs, and a license is just $50.

Step 2: Catch a redirect to the app store by navigating through your app

This is the hard part. Mobile app store auto redirect ads are typically frequency capped, targeted in obscure ways to avoid detection, and are difficult to replicate. If you’re having trouble, see the Advanced section at the bottom of this article for ways to speed up the process. Make sure you leave Charles Proxy recording your traffic, since you’ll search through the results to find the root source of the mobile app store ad. I’m not trying to pick on Candy Crush here in particular either, they’re just a popular example these ads point toward. (more…)

Charles Proxy on Cellular Networks

December 13, 2016

Note: Special thanks is due to Scott Eichengrun for this article, specifically showing me how to get the two phone rig going.

This article is a tutorial on how to configure Charles Proxy to inspect traffic over cellular networks, and while it’s designed with ad operations use cases in mind, it’s applicable to any front end web developer with similar needs.

There are many articles out there on how to use Charles Proxy through your phone – I’ve written two myself, in fact. All those articles assume you’re leveraging a Wifi network when connecting to the internet, however and while that’s fine for basic testing on mobile web and mobile apps, there are often reasons why you want to inspect traffic over a true cellular network instead.

Perhaps you need to debug an ad that relies on carrier targeting, or you want to measure data usage or network latency through a true cellular connection. These are more advanced use cases to be sure, but when you can’t get by with using your phone with Charles over Wifi, or a mobile emulator in Developer Tools. Plus, Charles Proxy offers a host of powerful features like breakpoints, which you can use to test an experience in stages, or rewrite rules to reference a local file before you update your server, or blacklisting, which can be helpful to isolate the root of various technical problems.

The Hardware Setup

Before you start, you’ll need:

A laptop running Charles Proxy. I’m using a Mac in this case, but you could do this with a PC as well.
Two phones, at least one which can be enabled as a mobile hotspot. I’m using two iPhone 7 in this case, but you could do this with Android devices as well.

Yes, unfortunately you’ll need two phones to setup this rig – the first is used as a mobile hotspot, the second to actually browse the web / app you need to test. The reason you can’t simply run the connection over a single phone is because you have to manually set the IP address and port of your network connection for Charles to inspect your traffic, and you can’t do that on your phone’s cellular connection. Creating a mobile hotspot however gives you the ability the adjust those settings on the device connecting through it. So you’re using one phone for its mobile network and the other phone as the client that proxies requests through Charles.

(more…)

Advanced Charles Proxy Tutorial for Ad Ops

December 13, 2016

I’ve already written a number of posts on how to use Charles Proxy for ad operations work, including how to setup Charles for mobile debugging, both over Wi-Fi and cellular networks. This post is an advanced Charles Proxy tutorial, meant to highlight best practices and advanced tips once you know the basics. I’ve suggested use cases for each that apply in the ad ops world, or that I use the tool for specifically.

Save / Open Charles Proxy Sessions

If you’ve ever been frustrated when trying to reproduce a technical error, being able to save your Charles Proxy session, send it to someone else, and have them open it on their machine to review the same issue is a godsend. I find this is especially helpful if you need to loop in the engineering team’s help, or send evidence of a particular issue to a vendor’s team. Charles can even open sessions saved in browser based developer tools as well, so you don’t necessarily need to pay for a Charles license for everyone on your team to benefit from this feature.

Multi-Session Windows

As a best practice, I also try to record each page load I render as a separate session, so I don’t get confused as to what happened on what refresh cycle with my browser. In this instance Charles Proxy easily outpaces a browser developer tool, which typically requires you to open a new tab if you want to load a new session and seems to consume a huge amount of system resources per session. For whatever reason Charles is much more efficient in the way it leverages RAM.

Another option on this front is to leverage the Auto-Save feature in Charles, which can automatically save your session every X minutes or so. Once Charles saves your history to a file, it clears that data from the active session, freeing up system resources. The multi-session approach makes sense if only need to test for a short period of time, while the Auto-Save feature is a better option if you’ll be testing lots of things over an hour or longer. For example, if you were validating an ad server migration across a number of websites.
(more…)

Charles Proxy Tutorial for Ad Operations

December 13, 2016

Charles Proxy is a popular application for web developers generally, also used in the Ad Operations field. Ad Ops staff use Charles to debug digital ads and ad technology like header bidding setups, ad server configurations, and so forth. What makes Charles Proxy useful is that it records the HTTP requests between a computer’s browser and all the different servers it actually interacts with to render a webpage. That includes communication which isn’t clearly visible in the source code of a webpage, but the browser encounters through redirects or items referenced or embedded in other scripts.

When to Use Charles Proxy

Charles Proxy is ideal for debugging when you can’t use the built in developer tools that modern browsers now offer, and I’ve listed some use cases below.

Testing mobile or desktop apps
Testing on cellular networks vs. Wifi
Altering request parameters on the fly
Replacing a server file with a local file
Refreshing specific elements on the page only

In my own view, Charles Proxy is more difficult to learn than developer tools, and can be a little frustrating in the setup work that’s often required. New users should expect some bumps on the way – be patient. So I don’t look to it as my first option, but when I’ve really got to dig in, there’s simply nothing better.

Setting Up Charles Proxy

Depending on what you need to test, you’ll have to do a bit of initial configuration to get Charles working. Typically, you’ll need to at least install Charles Proxy’s root certificate on your laptop, and if you plan to test mobile traffic, on those devices are as well. I’ve written a thorough walkthrough of how to install Charles’s SSL certificates on your laptop and phone here (for Mac), as part of another article on how to use Charles over a cellular network. (more…)

Adobe’s Vision for Ad Validation: Project Adthenticate

November 30, 2011

While new to the market and perhaps less established than AdValidation, Adthenticate is an exciting development in the ad validation space for lots of reasons. First, it has the resources of Adobe behind it, a mammoth corporation with some seriously smart development talent which I hope will continue to build on the current offering. Second, Adobe owns Flash, the mainstay creative format of virtually every form of desktop display rich media ad, which means it has more than its fair share of QA problems for publishers, and for which Adobe is best positioned to address. Adobe understands this technology better than anyone else possibly could, so it’s exciting to see a technology owner enter the validation space. Finally, after speaking with Adobe directly, it’s clear they have a forward thinking vision for where this technology can go, the potential applications, and the resources and clout to make it happen. (more…)