Interactive ads are everywhere these days, but when it comes to the technical process of getting an ad on the page and how publishers and marketers verify it delivered, not many people outside ad operations can explain what actually happens in detail. Read this article though and you’ll be one of them! Below I’ve detailed step-by-step how a browser gets from the initial call to a publisher’s website to the final ad creative, and when and how each party counts an impression. You can view a diagram of the ad serving process at the bottom of this post – the numbers in the text refer to the steps labeled in the diagram.
So, without further ado –
Calling the Publisher Ad Tag
When a browser navigates to a publisher website (1), the publisher’s web server sends back a bunch of HTML code (2) that tells the browser where to get the content (3) and how to format it. Part of the HTML code returned to the browser (4) will include a coded link known as an ad tag.
Here’s an example of what an ad tag from Doubleclick, one of the major ad serving companies, looks like:
http://ad.doubleclick.net/ABC/publisher/zone;topic=abc;sbtpc=def;cat=ghi;kw=xyz;tile=1;slot=728x90.1;sz=728x90;ord=7268140825331981?
Click here to read more about ad tags and how they are constructed.
The ad tag points the browser to the Publisher’s Ad Server (5), a system designed exclusively for delivering and tracking advertising. In most cases, the Publisher’s Ad Server is actually a network of cloud servers owned and maintained by a separate company. In this case, the content server tells the browser to fetch the ad from Doubleclick, a company owned by Google, that then makes the very complex decision on which ad to serve using a program called an Ad Selector.
The Ad Selection Process
In many cases the ad server is deciding among thousands upon thousands of potential options in mere milliseconds. The computational power behind the Ad Selector is mind blowing – these systems handle tens of billions of decisions a day and trillions in a year. The Ad Server makes a decision, and in most cases sends back another ad tag (6), or redirects the browser by pointing it to the Marketer’s Ad Server. These redirects are technically speaking 302 redirects, which tells the browser the page has been “temporarily moved”. This allows Ad Servers to count the 302 call as an impression and host the actual ad content on a different server. Once the publisher’s adserver sends the browser a redirect to the marketer, it counts a delivered impression in its own database (star). The only exception here is if the publisher decides to deliver a house ad or the marketer has asked the publisher to “site-serve” the ads, both of which requires the publisher load the actual creative files into their ad server, meaning the publisher is the final destination, and the browser can skip the loop through the marketer side (steps 7,8,11,12).
Click here to read more about why Publishers and Marketers have their own Ad Servers.
Calling the Marketer Ad Tag
The browser now calls the Marketer’s Ad Server (7), and is redirected yet again to a Content Delivery Network, or CDN, (8) a global network of cloud servers that actually house the raw creative graphics to fetch the actual ad. Why, you ask? Well, as powerful as ad servers are, they just aren’t equipped to handle the volume and bandwidth required to deliver content as heavy as image files. Redirects are often nothing more than a 1×1 pixel requiring just a few bytes of memory. Image files on the other hand are kilobytes or even megabytes in size, could be called millions of times a day, and require a much faster and robust infrastructure. Ad Servers might maintain three to six data centers across the world, but a CDN can process the heavy bandwidth and deliver the content faster because they operate hundreds of data centers and can route requests to the one nearest to the user, no matter where they are on earth. You can think of the ad server as the brain and the CDN as the brawn. Ad Servers aren’t the only companies that use CDNs, in fact many websites host their bandwidth intensive files in these cloud networks. A CDN is almost always another independent company, such as Akamai, that hosts the heavy creative assets so the Ad Server doesn’t have to. There used to be a handful of these companies out there, but Akamai has acquired almost all of them and is the largest player by far in the space.
Here’s what a CDN redirect to an Akamai server hosting a flash file looks like:
http://spe.atdmt.com/ds/ABCDEF12334/filename123_300x250.swf
In addition to sending back the redirect to the CDN, the Marketer’s Ad Server also appends a second redirect (10) back to itself with a query string to fetch a 1×1 pixel (11) after the ad content has been called. When the browser fires this last redirect calling a 1×1 pixel from the Marketer’s Ad Server (11), the Ad Server knows the ad was successfully downloaded and it finally counts an impression in its own database (star).
In many cases, your browser has to make at least four calls for site served ads and six in the case of third-party served ads for this whole process to work, if not even more, but shouldn’t take more than a second regardless of the number of parties involved. To visualize the process explained above, please see the diagram below – 302 redirects are highlighted in blue, and the ad creative is highlighted in red.
For more on the technical process of ad serving, read how real-time bidding works and how mobile ad serving works.
Brillant description. Concise and effective. Thank you !
Hello,
Good explanations indeed
Can you explain clickcount in detail as well?
Is it ; publisher adserver -> marketer adserver -> site ?
Or ; marketer adserver -> publisher adserver -> site ?
How clicktags in flash are handled ?
Thanks !
Hi Matthieu,
Clicks are tracked in much the same way as impressions, with redirects. Typically the publisher’s ad server populates multiple click tags into the href of the ad, concatenated with a query string so that when a user clicks the ad, the browser calls all three links more or less simultaneously. But yes, the specific order with any 3rd party system is always publisher ad server, marketer ad server, final destination. Highly simplified for sake of readability, the href to track clicks on a standard .gif ad might look something like this: href=”http://ad.doubleclick.net/activity;id%123456789%redir%http://marketeradserver.net/click;id%123456789%redir%http://www.destinationurl.com/”. Flash and javascript ads approach how the code functions slightly differently, but the method of using redirects to count the same action in multiple databases is the same.
Hope that helps –
Ben
Brilliant website indeed! It’s especially useful for people like me who are very new to this industry and working as ad-ops engineer. I’m still trying to learn the ropes of this industry and this website has been a great help to me, a life-savior almost 🙂
Thank you. And continue with the great job!
Awesome, fantastic i really enjoyed while reading this and clear all my doubt and comcepts how an ad is delivered while an ad call is made….thanks Ben
I’ve been wondering how ad servers worked, and this is by far the clearest explanation I can find. Very well written, thank you, Ben!
Job well done, Ben. Thanks for posting. This is one the clearest descriptions of the overall process I’ve seen.
Great article! Please I have 2 questions:
1. If a marketer has asked a publisher to “site-serve” the marketer’s ad, can the marketer still use a marketer ad server to verify the impressions and clicks delivered by the publisher?
2. If a marketer is using a marketer ad server to serve their ad, can they still use a second marketer ad server to verify the impressions and clicks delivered by the first marketer ad server?
Hi Gbolahan,
Thanks –
1. I suppose so – they could ask the publisher to host the creative, but still embed some type of tracking pixel in the ad. I’m not sure why anyone would do this though – usually creative is site served because the advertiser is too small to have a need for their own ad server. Perhaps if the publisher was responsible for developing the creative on the advertiser’s behalf, they would host the creative, but still embed some kind of 3rd party tracking mechanism for the advertiser. Both the publisher and advertiser ad servers will end up hosting the raw creative asset on a CDN like Akamai, so there isn’t a benefit from a latency standpoint.
2. They can, this is called 4th party tracking and tends to happen when the marketer will have a handful of rich media ads that are served through a different system than their primary ad server. In this scenario, the rich media vendor would be the 3rd party from a publisher perspective, and the primary ad server, perhaps DART, would be the 4th party. Generally, publishers do not allow advertisers to bill on a 4th party tracker because they incur a dual discrepancy, but the practice of using them is quite common to centralize reporting.
Hope that helps –
Ben
Thanks a lot for your reply Ben. I’m clear now.
Loving your write ups, quick question.
With regards to how ad.serving works between the marketer’s ad.server and content delivery network, does the marketer send a 302 redirect to the content network via the browser or is it a different type of redirect?
As in you say once the 302 redirect has occurred the impression is counted for the ad.server. But in the case of the marketer, the impression isn’t counted till the actual ad is served (having bounced the query pixel off the browser and back) which is after the initial redirect of browser to content network.
Cheers,
Allan
Hi Allan,
Yes, the ad server tells the browser to fetch the creative file from the CDN through a 302 redirect. If you were to view a page through a web development tool like Firebug, you would be able to look at both the HTTP request and response headers for each piece of the ad call, and you would see that the initial call starts with a 200 request, or a standard GET request to the publisher’s ad server, that call responds with a ‘302 ‘Object moved’ header, and the location of that file, redirecting the browser to the marketer’s ad server, which responds in kind with a 302 ‘Object moved’ header, and the location of the file on the CDN, which it can finally fetch.
To be clear though, it’s usually only that simple with simple .gif or .jpeg ads. It can get a little confusing in practical application because the process may not look so straightforward if, for example, the tag loads a flash creative. In that case, the tag may redirect to a script, and the confirmation pixel via a 302 response, and the script may have the subsequent calls to the CDN already embedded in the file. In that case, it isn’t technically a 302 redirect, but likely just looks like another 200 GET request. You can start to wander down the technical rabbit hole pretty quickly here, so my goal was to explain the process as simply and in a way that was as easy as possible to understand. To really dig in, I would recommend you speak to a sales engineer at your ad server, who will know the specifics for your setup with their technology.
Hope that helps –
Ben
Ben,
I built a .net site for a client, and created a custom ad rotator that logs click throughs to a database and then redirects the browser to the target url. It works great for all the jpg’s in the rotator, EXCEPT for a Doubleclick.net ad. I can log the image click, but can’t response.redirect to the target url. I just get a “page can’t be displayed” error. I’ve read that Doubleclick ads need to open in a new window, and tried several variations of window.location… and window.open(), but they go nowhere. Any help would be greatly appreciated. The ad url is below, and the TimeStamp var correctly inserts the proper format of the date, as required in the url.
http://ad.doubleclick.net/jump/N2886.291385..COM/B5246111;sz=190×190;ord=?”
Thanks very much!
Bob
Hi Ben,
I find your explanation extremely interesting, I was really looking for something like that.
I would like to ask you other informations, in particular I’m focusing my attention on CPA: CostPerAction. I know we can divide it in different subcategories: CostPerLead, CostPerSale, CostPerEngagement. For example, I don’t want to pay just because someone arrives on my landing page by clicking on a banner, but I want to pay if someone arrive on my landing page and complete a registration to my web site (CPL) or, even better, if he complete a purchase (CPS). How can the ad server “understand” that the customer has completed a purchase? How can it know that he performed a registration?
Another new type is CostPerCall, especially in the mobile area, where an advertiser pays when someone calls, for example, a call center. I have seen different apps that, after a click on a banner, automatically redirect you to your phone app, with the number already composed, you just have to press the “call” button. But my question is: how can the ad server be sure that I complete the call process? How can it know that I have not pushed the “cancel” button?
Thank you very much
Cheers
Alex
Hi Bob,
Thanks for your comment – I’m not sure I’m technical enough to understand what you’re trying to do, but it sounds like you’re trying to append your own click tracker on the fly to an ad tag by standing in the middle of the redirect path? That particular doubleclick tag won’t open for me, so there’s not much advice I can give you at this point without seeing more details, and even then I’m not sure I have enough development skills to really help. You might try looking through the doubleclick documentation online, or connecting with a broader forum of Ops people at AdMonsters, which has a free forum where you can ask questions.
Sorry I couldn’t be of more help – good luck!
Ben
Hi Alex,
Thanks – ad servers track conversions, however you want to define it, through pixels. Effectively, you would get a 1×1 image file from the ad server that you would place on the confirmation page for your conversion – this might be a thank you page or an order summary page, whatever destination would confirm the user actually took the desired action. Now, when a user converts, they land on that page and calls that pixel, which tells the ad server they converted, and allows the ad server to associate the right media to that user by looking at the cookie it has on that user’s machine. It’s effectively the same process used to track an impression, just without the ad itself involved in the process. You would have to do a little extra work in trafficking the ad to link everything up correctly (I would refer you to your ad server’s documentation for the specific details that apply to you and your ad server configuration), but it’s fairly straightforward to do, and a common strategy, so it should be well documented.
In terms of cost per call, I believe that is exclusively a search concept vs. the display ad concepts I tend to focus on here, but as I understand it, there are a few ways to do that, depending on how you plan to execute. For example, Google’s bid per call in the desktop environment requires the advertiser to use Google Voice generated tracking numbers (usually positioned to the user as an extension), which allows it to track the conversion. For mobile devices, I’m less familiar with the approach, but I imagine it works either through a click tracker that fires when a user clicks on a phone number in a search result, or also depends on a Google generated call tracking number. Google is the only company I’m aware of that actually offers this type of service.
Hope that helps –
Ben
This is great, thank you. Do you know if there are best practices as a publisher re: working with multiple ad exchanges? Should i be calling each exchange concurrently or sequentially based on their response?
Thanks.
Hi Ben,
First off, excellent job on this article. I’ve searched high and low for simple explanations of ad serving and this is by far the easiest and most intuitive read yet. Second, I had a quick question about using fourth-party calls in Google swf banners being served up via the Google Ad Network.
In short, my banner swf file is dynamically calling on three separate jpg files using absolute paths (i.e. http://www.mydomain.com/frame1.jpg). These banner frame jpg’s are hosted on our own internal server so that we can update the banner creative on demand “without” having to edit, change, or resubmit new swf files to 3rd party vendors.
The problem I’m running into is that when I upload my banner swf into the Google Ad Network I get denied as they don’t permit 4th-party server calls. Is there any sort of workaround for this by chance? We still want to use the Google Ad Network for banner tracking ect (clickTag) but on the other hand we’d also like to host our own banner creative. Any advice is much appreciated and thanks again for such an awesome article.
Best regards,
NM
Hi Chris,
Ideally, you would call each exchange concurrently, and take the highest paying bid from either. Without an SSP to sit as a layer between the exchanges and help you manage this process, I’m not sure exactly how you would do this. If you don’t have the scale to make it working with an SSP make sense, my advice would be to try and test a few exchanges at the same time and see who can pay the most. The best test setup would be to run rotate a cookie value on your users for however many exchanges you want to test, and silo the exchange calls per cookie value so that each exchange gets the entire user session. If you mix impressions between exchanges from the same user session, you’ll confuse the frequency tracking of buyers, and potentially throw a wrench in your final results.
That said, my guess is that if you execute the test successfully, you’ll see similar results from every exchange, since most bidders are on all the exchanges, so it shouldn’t really matter all that much which one you pick.
Good luck!
Ben
Thanks NM,
Probably not – Google Engineers are pretty smart – they’re not likely to allow a loophole, at least not for very long. That said, Google does allow some advertisers to 3rd party serve, and that would pretty much solve your problem http://support.google.com/adwordspolicy/bin/answer.py?hl=en&answer=94230. With a 3rd party tag, you could swap out the display creative whenever you want, without having to change the actual ad tag in the Network. Google says this functionality is available only by invitation, but you could ask support if you really wanted, and probably find a way to get there as long as you can work with an approved 3rd party. Of course, this probably doesn’t make sense unless you are a big advertiser. If you’re a small shop, I doubt there’s a workaround. This is really a question for Google Support – I’m just not this intimately familiar with the terms and policies enough to give you a solid answer here.
Best of luck!
Ben
Ben,
I have read your 3rd party ad server explanation with great interest and as a marketer, found it very helpful. Thank you.
Question: If I chose to work through only Doubleclick as my ad server am I limited to a finite number of sites that Doubleclick has contractual relationships with or would I be able to have my ad served up to any website(s) I decide on? So as a logical extension of that same thinking does Atlas or iBlaster similarly have their own finite (or favourite) set of websites they can deliver ads to?
Let me know.
Thanks,
Tim Goudie
Ben,
This piece of content is fabulous. But something which still keeps ringing is why Akamai (or equivalents). What would the possible issues be if I as a publisher wanted to host ads for my customers, taking images and videos as inputs. I have complete control over the content. I can host these content on my superfast cloud based content server with no need having to carry out 9,10. I can pass the storage charges to my customers.
Could you please provide in realistic terms, what savings we derive out of using Akamai et.al.
Many thanks and look forward to hearing more in the days t come.
Hi Tim,
Glad to be of help – to answer your question, no, any ad server should be able to serve on any webpage. Ad servers provide publishers with a very simple technical mechanism called an ad tag which is easy to implement on any website, even if the ad server has never served on that site before. In fact, in the majority of cases, the ad server won’t have any direct relationship with the publisher running its tags, and in many cases may actually have competing interests. Plenty of Atlas-serviced advertiser tags are running on Doubleclick serviced publisher site right now, for example, and you can repeat the scenario with virtually any combination of ad servers out there. The only exception I can really think of would be from an ethical point of view – most ad servers wouldn’t knowingly let you run ads on adult content, pirated content, hate speech content, or things like that, but I assume that’s irrelevant in your case.
Hope that helps –
Ben
Hi Surojit,
Thanks – Akamai, or any CDN for that matter isn’t necessary per se, but does help reduce latency. Akamai has done the very difficult and cost intensive work of setting up and maintaining physical data centers around the globe so that content loads extremely fast to a user. See, the ad server can make quick decisions for any user from only one or two geolocations because it only processes a few bytes of data before passing the user on to a CDN based service to fetch the actual content, which might be thousands or even millions of times larger than what the ad server needs to process. It makes more sense to have a service that keeps the heavy data physically closer to a user, and on a system that is setup to load balance for much heavier volume. The CDN doesn’t have to make any variable-based decision, it just has to provide access to the file quickly.
Practically speaking, unless you were to setup your own ad server, I don’t know that you can get around using a CDN – the ad server just automatically moves it onto those servers for their customers, it’s not as if you have a choice from a customer point of view. But even if you were to build your own ad server from the ground up and decide to not use a CDN, it’s hard to say what your cost savings would be. At the end of the day, you still have to carry out the service that steps 9 & 10 provide, which is deliver the creative to the user, and verify that it fully rendered on the page. So I don’t know that you really save any steps, you just change where those steps happen.
The question to you would be will you not geolocate your data service at all – if so, what is your cost on the user experience and your system performance to customers? You’ll almost certainly have an inferior product to your competition. If you will geolocate your data service, why bear the entire cost of that yourself, when you could just outsource it to a company that provides for that need as their primary service? It would seem to not make economic sense to me to go about it that way – I would expect it would actually cost you more not to use a CDN service like an Akamai.
So that would be my take – hope that helps, and best of luck!
Ben
Hi Ben,
Could you clarify where a digital media agency would fall in the ‘Ad-serving’ process you had elaborated. I remember seeing, planners dealing with the Publishers directly, and not with marketers(or advertisers); nor had I ever once heard them talking about marketer ad-servers. Does the process work w/o marketer ad-server’s in picture?
Please do clarify.
Thanks,
Anish
Hi Anish,
Sure – basically the chain of events is as follows:
1. User calls a publisher – publisher’s web server redirects the user to the publisher’s ad server
2. User calls publisher’s ad server – publisher’s ad server decides what ad to serve, and redirects the user to a marketer’s ad server
3. User calls marketer’s ad server – marketer ad server knows what ad to serve based on the incoming call and delivers the ad to the user (or more likely redirects the user once more to a Content Distribution Network like Akamai, which hosts the actual creative.)
The agency is the marketer in this case, or is the marketer’s agent. If you, a media planner, work at an agency, the marketer is your client, and the company actually paying for all the advertising the agency is buying. There’s always a marketer, they may just be an indirect participant if they’ve hired an agency to handle their advertising campaigns. I think it’s just a case of semantics – when I say marketer, I really just mean the buy-side of the equation – that could be an agency working on behalf of a marketer, or it could be a marketer directly, as some do their own media buying and don’t contract with an agency.
Hopefully that makes sense –
Ben
Ben,
Excellent! Really glad you took this initiative.
Could you help clarify my understanding. This is what i think happens
1) Publisher AdServer will only determine that the add needs to be served for a specific advertiser. But we dont know what ad at this point.
2) One the request is re-directed to marketer’s (advertisers) ad server, at this point the creative is identified that gets served. And the location to fetch the creative is sent in the redirect. The location is typically a CDN (Akamai) path.
3) User browser does the final lookup to locate the most nearby CDN server and load the add at this point.
Is the above correct?
If for some reason, ads were loading fast, and suddenly they start to load very slowly, what could be the potential bottleneck?
When a publisher approaches a network like doubleclick for ad serving, does it always do a redirect to advertiser ad server, or can it serve the ad itself? And if it can serve the ad itself, how is the pricing for advertiser determined.
Thanks again.
Thanks Chintan – let me see if I can answer your questions:
1. Right, the publisher decides what client needs to serve in order to fulfill delivery quotas based on the goals and campaigns in their system, and for what the incoming ad request qualifies.
2. If a 3rd party tag is used, that’s correct – the marketer will determine which creative actually renders on the page, which they house at a CDN. If a publisher’s house ad or locally hosted ad is served, the publisher will simply redirect to the CDN themselves, where the raw creative is hosted.
3. When 3rd party tags are used, correct – and for most paid ads on the internet, 3rd party tags are used, so your understanding is spot on.
4. If the ads were loading slowly, there could be a few reasons – first, there could be more parties involved in the ad serving process. So, if the publisher redirects to a network or an exchange, there will be additional latency. Alternatively, it could be a matter of lots of pixels involved in the serving process – for example if a verification script like AdSafe is used to scan the page before the ad is allowed to load. Finally, it could also simply be a matter of bandwidth on the user side.
5. The publisher can host and serve the creative themselves – this is the difference between 3rd party ad serving (when a publisher redirects to a marketer’s ad server) and 1st party ad serving (where the publisher hosts everything themselves). Usually the pricing is the same regardless of the setup – it doesn’t really save the marketer or publisher any money to have it setup one way or another unless the marketer doesn’t want to serve their ads at all, in which case they don’t need an ad server and I guess would save that money. It’s too painful operationally for a large organization to do that though, and would actually be more expensive to try and keep track of reporting, etc. For small marketers it might make sense, but it won’t save them all that much money – ad serving can be had for extremely cheap these days.
Hope that helps –
Ben
Once again, brilliant. Thanks alot!
Hi,
Thanks for the explanation!
I am working on a website that dynamically generates content, from user’s queries. There are literally billions of different pages that can be generated, each with a set of associated keywords (constructed form the queries and the relevant content). For example, a page might have the keywords “kitten cat feline mammal”, or “judge jury sentence law” etc.
I want the appropriate adverts to be served, such as for cat food products, or lawyers, or whatever, depending on the keywords. The site cannot be spidered, as all the pages are virtual, and dynamically generated. (Kind of like a search engine, where Google results pages have related adverts.)
I am having great trouble finding anywhere that can do this, and have met several dead-ends, where I explain and explain, and finally they say…”Ah!… No, we cannot do that.”
If you have any advice or pointers, or industry terms that I should know, that would be very helpful!
Thanks,
Mike
Hi Mike,
I’m still not sure why this would be a deal breaker – plenty of sites dynamically generate content and their URLs. Perhaps I’m not fully grasping what you’re trying to do, but there’s nothing new about dynamic pages. Your problem could be if you use the same URL for multiple content iterations, in which case, I’d suggest you change your approach and just dynamically generate a unique page URL along with whatever content you have. This should be fairly straightforward for someone with experience coding PHP.
But all that said, you still need someone with the demand liquidity and breadth to serve relevant ads on the fly from everything from cat food to lawyers to take your examples – only the exchange, or Google Adsense can provide that kind of granular relevancy, and really only Adsense is that focused on the context. The exchange will care more about the user data, regardless of what content they are reading. So I would say there’s no reason you can’t get Adsense to work the way you want – you may just need to change your coding approach somewhat. Read this: http://support.google.com/adsense/bin/answer.py?hl=en&answer=20763 As long as you fill your impressions and generate revenue, is it really that critical how the ads do that? My advice would just be to read up on how Google works with dynamic content, and ensure you follow their best practices.
Good luck!
Ben
We have been running an ad tag and dont see anything in the reporting. What could the issue be? Thanks.
Hi Marcia,
It’s very tough to tell – there could be all kinds of reasons. I would try asking your ad server for help, or you may find this article on identifying and fixing report discrepancies helpful: http://www.adopsinsider.com/online-ad-measurement-tracking/resolving-3rd-party-discrepancies/
Ben
Ben, this is by far the best explanation of how this entire process works that I’ve been able to find. Thank you so much for all your insights and advice.
I am trying to further understand the process regarding the how the payments and timing of when a “publisher” (website operator) gets paid for the ad revenue they’ve generated off their site. I know there are many dimensions to it, but here are my basic questions that perhaps you can help with:
– I understand that both the marketer and publisher ad servers keep track and ensure a successful “impression”, but how to the parties reconcile the numbers if each is reporting something different?
– How is the tracking of the “billing” done and ultimately, how does the marketer know which publisher to pay — and how do they “pay”/transfer the money to the publisher? I think if this is done via Google’s network then the publisher gets paid via their AdSense account, but how does Google in this case “bill” and collect from the marketer? How does it work if not via Google?
– How long does it take for a publisher to get paid and what contractual commitment is in place to ensure the publisher can pursue the marketer if they refuse to pay?
Thanks so much again for all your insights to how this works!
Hi Tom,
Thanks – to answer your questions:
1. Reconciling discrepancies is something covered in the IAB’s standard Terms & Conditions, which is in version 3.0, Sec XII.D. You can read it here: http://www.iab.net/guidelines/508676/tscs3 It basically says that the Publisher and Advertiser will make a good-faith effort to resolve any technical problems should they arise, but that the Publisher should be able to bill in full if the reporting systems are within 10%. In practice, that’s not what happens – instead, to keep their relationship with the Advertiser on good terms, Publishers usually add what is called an “overrun” and basically bonus every campaign 10% on top of what was contracted to ensure the Advertiser’s server reports full delivery. Since this is the common practice, expect advertisers to balk at the suggestion they pay in full if their server hasn’t reported full delivery. It is on the publisher to check the advertiser’s reporting and ensure there isn’t a major technical problem. You can read more about resolving 3rd party discrepancies here: http://www.adopsinsider.com/online-ad-measurement-tracking/resolving-3rd-party-discrepancies/
2. So for Ad Sense, Marketers have to pay upfront, they basically put a credit in their Ad Words account that runs down as their campaigns run. So billing is pretty easy compared to most agency deals – publishers for their part each have a unique ID in their tags, so that when the tag makes a request for an ad to Google’s servers, it passes an ID that Google can log in order to attribute the revenue to them specifically. Google then wires the money to the publishers bank account, which they’ve had to link and verify in their AdSense account on a monthly basis, assuming they can accrue more than $100 in revenue that month. If they can’t, the money just rolls over and waits until the Publisher meets that threshold, at which point a payment gets sent. In the traditional Publisher to Agency or Advertiser direct sale, a contract called an Insertion Order, or IO is cut. Check out the IAB document, Section III. Usually bills get sent once a month by the Publishers for the prior month’s activity. So on Oct 1st, lots of publisher will log in to the Advertiser’s reporting portal, pull the 3rd party reporting for the campaign activity for September, and will use those numbers to generate invoices. Sometimes agencies request specific payment terms, perhaps they want to be billed evenly, regardless of actual delivery in a given month. That is, if you had a 6-month campaign for $60,000, you’d just bill $10,000 each month, even if you delivered let’s say $10,108.33 that particular month and $9,891.67 the next. The IO should specify a billing contact for the Advertiser.
3. That’s all in the IO. The IO is a legal contract which should append the IAB Terms and Conditions I mentioned above, which states that all payments should happen on Net 90 terms, meaning the Advertiser has to send payment within 90 days of being billed. Many agencies are notoriously late with payments, usually because they are also waiting for their clients to pay them for the work. In certain cases, Publishers will suspend all campaign activity if a large balance builds up and the agency stops paying, but this is rare. In many cases, Publisher know the agency is playing the waiting game themselves, and extends a certain amount of goodwill for important clients, who they know will eventually pay up. If you can’t get paid though and you have a signed IO, you can pursue legal action, or refer the case to a collections agency.
Hope that helps – good luck!
Ben
Thanks Ben, very helpful … I appreciate the links you included for finding out more.
Thanks!
Tom
Say if I was not accessing a website but using a piece of software installed on my desktop which I was logged in to and had ads served to…
Would anyone in the process have an identifier that the ad was served to me/my computer/user account?
Who in the process outlined above would have this?
Hi Ed,
It’s hard to say without seeing the exact program, but I wouldn’t expect it would be all that difficult to assign a unique ID to you. After all, if the software is serving ads, it must be connecting to the internet to contact an ad server, and probably has some kind of browser based functionality which would allow it to assign a cookie to you. It can deliver the ad back, so there’s no reason it shouldn’t also be able to deliver a cookie back. In this scenario any ad server could assign you a cookie – either the software, or the software plus a 3rd party ad server (if one is in use). Again though, it’s hard to know how it might work for your specific program.
Ben
Great article. Very Informative ad serving information
Great article. Nice descriptions on ad serving
Hi Ben,
Great work mate you are a true legend!
So according to you which 3rd pary ad serving company is good?
I am currently in the process of developing my website to facilitate online trade and in the process would like to publish few banner ads on my website to (a) generate traffic and (b) generate revenue
Cheers
Sidd
Hi Sidd,
There seem to be fewer and fewer ad serving options out there for small publishers (let’s say under 1MM imps / month) – in fact, I’m not sure there’s really a compelling solution outside of DFP Small Business at this point if you just need free ad serving at low volume. One of my favorites to recommend, AdZerk, just got out of this business, and some other solutions have gotten out of this service as well. But, if you’re going to be doing heavy volume I’d recommend you take a look at them. But, if you are at lower volume, unless you are selling multiple, dedicated campaigns that all have impression goals that need to be met, or require different targeting, you might find that simple AdSense is your best solution to make money with digital ads.
Hope that helps –
Ben
Hi Ben,
Do you feel like niche Ad Serving companies like Pointroll and Flashtalking still serve a purpose when the big dogs are developing their own in hourse servers which speak to all the other bits they own? In terms of careers, Would you say DSPs and Trading Desk are the place to be as they’ll all l eventually have their own, if they don’t already have their own in-house server, developement, etc.? I’m considering moving from a trading desk to a niche ad server and these are my dilemmas. But I feel that these niche ad servers will provide a new set of skills I’ve yet to explore. It’s big Data meets big creative. There’s lots to learn in the ATD space but For me, it’s becoming more of a media planner position…similar to media agencies…just a bit more in-depth. And I’ve already done the agency thing. But yet working with a Point Roll or FT is a bit of a new world. I want to learn as much as I can and the server side of creative sequencing and such is something I’m still looking to learn. Let me know your thoughts.
Hi Winnie,
Tough question, though the niche ad servers you mention are more interesting I think from a dynamic creative perspective. That is, they are interesting and important in the way they allow advertisers to test and optimize messaging, and programmatically customize that messaging. If DSPs / Trading Desks enable smart targeting, the companies you mention enable smart messaging. I think the buying piece gets all the press, but the creative aspect can be equally powerful. That said, from a career perspective, the buying piece will likely afford you more opportunity since there are many more jobs and potential jobs going to open for people who are experts in the ATD / DSP space.
If you feel like you already have the ATD / DSP experience and knowledge set, then I would say creative / messaging would be a smart thing to learn next, or to try and expand your knowledge across channels. For instance, if you know about programmatic display, go find a place where you can learn about programmatic video, or mobile, etc. They are different worlds with unique challenges.
Best of luck in your search!
Ben
Hi Ben-
Really great post. This is a fantastic resource. Thanks for putting in the time to write this up and share it with the world!
In the case of having a media partner who will NOT be able to use an ad server at all, what are the options to enable tracking? Our company has a media partner in Korea that has informed us that they will be tracking media efforts without use of an ad-server. Everything there will be site-served. I’d like to get an idea of what our potential paths could be to try and measure that effort. Our goal is to not only measure the basic clicks, impressions, etc but also to measure conversions/transactions/sales on the client’s site as well when users land on the client’s site and make a purchase. Appreciate any tips, resources or workarounds that you may know of.
Hi Jo,
Just because your media partner isn’t using an ad server doesn’t mean they can’t serve 3rd party tags so you can track & measure the campaign on your side – they’ll just hardcode them on the page’s source code instead of dynamically serving them with a redirect. The benefit of an ad server to the publisher is not to provide tracking to the advertiser, it’s to easily target, rotate, and pace many different ad campaigns at once. Usually site served means you the advertiser have decided not to use 3rd party tags, therefore the media partner is hosting your creative in their ad server for you. But, if you have these tracking needs you won’t be able to get around using a 3rd party tag, especially if you want to measure conversions, which will require a cookie or device ID of some sort in a mobile context.
I’d try to find out from your media partner if they just don’t know how to post your 3rd party code or if they are refusing to use it because of some kind of policy. If it’s the former situation, you can probably talk them through it (you should pretty much be able to post the ad code directly in the page source code unless you have some kind of fancy video ad – that’s essentially how the AdSense tags on most small websites work), and if it’s the latter, well you’ll have to determine if tracking is a dealbreaker for you and if it’s worth running the campaign if you can’t track it.
Best of luck –
Ben
Hi Ben,
I think my initial comment got rejected, so I’m posting it in its original form here again:
Thank you for creating this post. I’ve referenced this article many times and I’d appreciate it if you could clarify the following points:
1. You write that “redirects are often nothing more than a 1×1 pixel requiring just a few bytes of memory” -> are redirects and pixel (requests) two different things from a technical/implementation point of view?
2. You also mention that the publisher ad server “makes the very complex decision on which ad to serve using a program called an Ad Selector” -> what is involved in the decision-making process? What variables/data does the ad server take into account and optimize for?
3. Finally, how would the diagram change if the ads were served in a an ad-network as well as a “blind” ad-network? If I’m not mistake, the process you describe above, there is full transparency in terms of where and which ads are served, correct?
Hi Igor –
1. Yes, a redirect is simply a 302 HTTP response code to a request, which enables one server to essentially step in the middle of the transaction flow to count a request. The 302 response allows them to redirect the browser to a new location, which can be another server that wants to count, or the final location of the file the user wants. In some cases, that file will be a creative image on a CDN, perhaps a 300×250 jpeg image. In other cases that file will be a transparent 1×1 gif. In that case, the purpose of the request to begin with is simply to cookie a user (perhaps for conversion attribution purposes, or that the user clicked on an ad, etc.) or count that the user was on a specific page, rather than to deliver an image.
2. The ad selector uses a number of criteria – I’d direct you to DFP’s ad selection whitepaper on this subject: https://support.google.com/dfp_premium/answer/1143651?hl=en
3. In the case of a network, the network would either take the place of the marketer ad server, or more commonly be an additional hop between the publisher and marketer ad servers, at least in a non-RTB transaction. In an RTB transaction, you’d have a diagram more like on this page: http://www.adopsinsider.com/ad-serving/diagramming-the-ssp-dsp-and-rtb-redirect-path/ In a non-RTB transaction, the network would “blind” the publisher domain simply by not making that granularity available in its reports to the marketer. Easy! In an RTB transaction, either the SSP (selling to networks / DSPs), or the RTB-enabled ad network itself (selling to DSPs) would strip the domain value from the bid requests they generate on behalf of the publisher.
The process I describe on the original article yes, there would be full transparency because the buyer has a direct relationship with the publisher and therefore knows what they are buying. In the RTB transaction model, the buyer may or may not have transparency, depending on publisher preference and configuration. That’s from a bidding perspective; from a reporting perspective, it’s very difficult to hide where an impression actually ran, because there are common technologies used by most advertisers today that can read the URL of the page their ads serve on once they deliver. So they can’t know in advance, but they can know after the fact.
Hope that helps!
Ben
Hi Ben,
Great article!
When a media agency buys through a vendor like Quantcast or OwnerIQ and gives the vendor tags but does not site serve and rather 3rd party serves the ad through the agency’s ad server, at what point does the decision on targeting and inventory with the vendor take place? Also why does the agency give 3rd party serving tags to the vendors?
Thanks!
Hi James,
In this case, OwnerIQ / Quantcast is likely serving as a media buying entity, most likely using a 3rd party DSP to buy media on behalf of a client against segment data that only they have. Both OwnerIQ and Quantcast are really in the audience data business, so this is likely a service they have available to simplify the process for buyers.
In this case, the agency, whether it is using site served tags or 3rd party tags is in no way involved in the ad serving process; the only decision they could possibly make is what creative to rotate in for a given impression. To answer your question on why the agency would use 3rd party tags in the first place, it basically comes down to simplifying the billing process, but you can read the full detail on that in another article I wrote here: http://www.adopsinsider.com/ad-ops-basics/why-do-publishers-and-marketers-have-separate-ad-servers/
Hope that helps!
Ben
Hi!
At my company we append both Double Click (raised by our Ad agency) and Query String Parameter (raised by us) tags to our digital inventory. The data for the QSP gets recorded in the Adobe Analytics tool which I access, while the DC tags data is recorded by the agency. The problem is that the number of clicks recorded by DC tags is typically 10 – 20 times higher than the number of visits. Now, some of the discrepancy may be attributed to people who clicked on the Ad, but due to page load time or realizing that a new browser tab is opening up, cancelled it before the page opened (our TTFB is quite fast, actually). But, clicks being 10-20 times higher than the visits does not seem to sit well. We have had our website checked for the backend DTM tags and there is no problem in data being recorded or its accuracy. One of the reasons we explored was the fact that if the same unique cisitor is clicking on a tagged link multiple times within 30 minutes, Adobe Analytics treats it as 1 visit. Hence, a doubt around bots arises. Could there by any other reason that you reckon might be leading to this?
Hm, hard to say without actually looking at the data, but my guess is there are a lot of ‘fat finger’ clicks that are unintentional. Here is what I’d do to debug:
1. Render your tag in a browser with nothing else on the page with developer tools open and see if the QSP is called in the Network tab when you click and visit. How long does it take? You may actually have to use something like Charles to record all the necessary activity of clicking between multiple pages.
2. Pull as granular a report as possible from Adobe / DFA – is one particular site / geo / size / device type / browser / etc have a much larger drop off than others? Can you isolate the problem to mobile for example, where users would be more likely to click by mistake? Or, are you sure this isn’t related to Ad Ops testing at the beginning of a campaign by clicking on the ad a bunch of times to test that everything works?
3. I imagine you can track both clicks as well as visits within Adobe because calling the QSP is different than rendering the Adobe js on your page. Are DFA and the QSP call more or less the same but the Adobe ‘visits’ metric is off? If not, you might have some kind of error happening on the append action.
That’s all I can think of at the moment to check – good luck!
Ben
Hi Ben,
This may be a dumb question, as I really don’t know much about ad serving (although this article certainly helped)
I was wondering if somebody could answer a couple questions I had about your ad serving platform.
Do ad servers (or any the content network) collect the full URL of the webpage that one of their advertisements gets displayed on? For example, I see (NOTE: I only see it, I don’t click it) an advertisement at the URL https:/www.youtube.com/results?search_query=cars. Do they record that whole URL, or just the fact that it was displayed on youtube.com?
I kinda feel like its a privacy issue when ad serving companies can collect so many search terms and things like that just because an ad appears on a site and collects the URL.
Thanks for your time!
Hi Kurt,
Sorry for the delayed response! To answer your question – yes, the ad server typically collects the full URL of the webpage so that publishers can target based on that structure. In your example, perhaps the publisher wants to let an advertiser serve an ad to everyone who searched for ‘cars’, and the URL is the easiest way to access that information.
From a privacy perspective, different people are comfortable with different types of tracking, and if that crosses the line for you in particular, you can stop that tracking by deleting your browser cookies on a regular basis or opting out of the tracking via the AdChoices program.
Hope that helps –
Ben
Hi Ben,
If you could help explain to me how a database would fit in to that equation. what would generally be stored in a sql database apart from Ip address and user name and post code.
I am currently trying to create a database to support an advertising platform.
Thank you so much in advance.
Ali
Hi Ali,
There are a number of databases you’d use in an ad server. You need a variety to hold metadata around objects like campaigns, orders, advertisers, and so forth. Then you need a database that would house the targeting for each campaign kept in memory so you can access it for ad selection, and of course a database to house delivery / performance data on disc for running reports.
I’m sure this is a 30,000 ft. view of the situation and a database admin would have a much more sophisticated POV on this topic but hopefully gives you the general idea.
Ben
Hi Ben,
Thanks for writing such an excellent article! :))
Based on the information provided here, I am trying to understand how advertising on video content or video platforms work? And I have few questions –
1) How does ad server detect whether to serve a video ad or banner ad?
2) How does video platform knows when to show an ad? (Pre-roll / mid roll or post roll), where is this defined?
3) Website has ad tags which gives call to ad server. But in video advertising, where the ad tags are defined?
4) I read through IAB video standards. But have not understood where exactly it fits in video ad serving workflow…
Could you please help me understand how video advertising works?.
Thank you so much for your time and help!
PJ
1) How does ad server detect whether to serve a video ad or banner ad?
The ad request will specify what kind of ad is accepted.
2) How does video platform knows when to show an ad? (Pre-roll / mid roll or post roll), where is this defined?
The video ad server has a way to configure inventory breaks and pods. Think of it this way, you might have three, 30 second ads that are allowed to run at a given break. The break it just one request that allows a pod, and the pod configuration defines how many ads / seconds are allowed to run.
3) Website has ad tags which gives call to ad server. But in video advertising, where the ad tags are defined?
The tags are still on the page, but the video player generates them. They don’t really sit in the source code of the page in the same way.
4) I read through IAB video standards. But have not understood where exactly it fits in video ad serving workflow…
Not sure what you mean – can you be more specific?
Thank you so much for the answers Ben! I really appreciate all your help and efforts.
For Q#4 – Apologies for the not specifying it clearly…
We have different Video Ad serving standards like VAST, VPAID, VMAP – I read through them, but I did not really understand at what point these standards play their role in Video ad serving process.
E.g. At what point in ad serving process VAST xml is generated? and Who generates VAST xml with all ad serving details?
Thank you again!
PJ