Charles Proxy is a popular application for web developers generally, also used in the Ad Operations field. Ad Ops staff use Charles to debug digital ads and ad technology like header bidding setups, ad server configurations, and so forth. What makes Charles Proxy useful is that it records the HTTP requests between a computer’s browser and all the different servers it actually interacts with to render a webpage. That includes communication which isn’t clearly visible in the source code of a webpage, but the browser encounters through redirects or items referenced or embedded in other scripts.
When to Use Charles Proxy
Charles Proxy is ideal for debugging when you can’t use the built in developer tools that modern browsers now offer, and I’ve listed some use cases below.
- Testing mobile or desktop apps
- Testing on cellular networks vs. Wifi
- Altering request parameters on the fly
- Replacing a server file with a local file
- Refreshing specific elements on the page only
In my own view, Charles Proxy is more difficult to learn than developer tools, and can be a little frustrating in the setup work that’s often required. New users should expect some bumps on the way – be patient. So I don’t look to it as my first option, but when I’ve really got to dig in, there’s simply nothing better.
Setting Up Charles Proxy
Depending on what you need to test, you’ll have to do a bit of initial configuration to get Charles working. Typically, you’ll need to at least install Charles Proxy’s root certificate on your laptop, and if you plan to test mobile traffic, on those devices are as well. I’ve written a thorough walkthrough of how to install Charles’s SSL certificates on your laptop and phone here (for Mac), as part of another article on how to use Charles over a cellular network.
Viewing Charles Proxy Results Walkthrough
To help get you started with Charles, I’m going to analyze the requests of one of my favorite ad-supported websites, cnet.com. Cnet writes news and reviews on everything from cars, phones, TVs, gaming platforms and other varieties of consumer technology.
Structure vs. Sequence View
When using Charles Proxy, the first decision you need to make is whether you want to see your results organized by hostname or chronologically. If you want to read results by hostname – that is, all the requests to ‘example_resource.com’ grouped together, select the Structure View. The structure view shows you the same information you’d get on the ‘Sources’ tab in Chrome Developer Tools. In this example, you can see there were two different requests made to 36a3fec2.mpstat.us, a domain owned by digital performance provider Soasta, Inc.
You can use the structure tab to quickly understand if you are making requests to specific domains and if so, how many and what they are. For example, you might want to check that you’re making a request per ad slot on the page for every header bidder you have integrated. This might be annoying to look at chronologically, but simple when organized by domain. However, if you wanted to understand if you were making all your header bidder requests before your ad request, you’d need that chronological view. In that case, you’ll want to select the Sequence View, shown below:
Once you select a specific result from either view, Charles loads up the details for that interaction, organized in a few ways. You can select the Overview tab to see a summary of all details, or one of the other tabs to view specific details like the connection timeline. I find that the Contents tab is by far the most helpful, because it splits the request parameters and response apart, making the URL details easier to read. It’s more or less the same as what you’d see clicking on a specific request in the Network tab if you were using Chrome Developer Tools.
In this case we see a request to our old friend, DoubleClick, with the usual parameters for ad unit, sizes, and so forth. If you wanted to understand basic information like what header bidder key-values you were passing, or all the eligible sizes for your ad slot, or the correlator ID for the request, this is your stop:
In the above, we were looking at the details of a ‘request’, but often times you also need to understand what comes back in the response to a call – like the bid value from a header bidder. While not well labeled, there are two panes on the Contents tab; the top shows the request, the bottom shows the response. You can visualize specific details for each by selecting from the respective tabs. In this example, there’s a request to a standard DoubleClick ad call above, and the response parameters of the line item that filled that request in the response pane below. This response shows everything we’d need to locate the exact object the ad server selected in DFP for this request.
I personally prefer using the Query String option for the request and the HTML or JavaScript option for the response, depending on what it is. A word of advice; if Charles does’t automatically offer a JavaScript tab based on the response details, you can tell it to apply that formatting with a simple right-click and selecting View Response As and then choosing JavaScript. I’ve found that for header bidder responses in particular Charles often doesn’t recognize the response format as JavaScript or JSON automatically.
In this next example, we can see a request and response to an AdTech (AOL) header bidding endpoint, with the request showing required parameters like placement ID, and the response showing bid values. If we were validating a header bidder integration, we’d be looking for that $2.26 value (shown in the ‘price’ parameter) passed to DFP through some key-value, which we could look for in the request parameters to the ad server, like we did in the previous use case:
Now you know the basics of how to use Charles Proxy to read ad request and responses, but the use cases are really endless. You could easily use Charles to parse a DMP’s response, optimize header bidder latency, QA an ad server migration, catch a mobile app store redirect ad, and more. Tell me in the comments your core use cases for Charles and read my best practices & advanced use cases for Charles here.
One comment
Comments are closed.