research-article

The ad wars: retrospective measurement and analysis of anti-adblock filter lists

Authors:

Zhiyun QianAuthors Info & Claims

IMC '17: Proceedings of the 2017 Internet Measurement Conference

Pages 171 - 183

https://doi.org/10.1145/3131365.3131387

Published: 01 November 2017 Publication History

Abstract

The increasing popularity of adblockers has prompted online publishers to retaliate against adblock users by deploying anti-adblock scripts, which detect adblock users and bar them from accessing content unless they disable their adblocker. To circumvent anti-adblockers, adblockers rely on manually curated anti-adblock filter lists for removing anti-adblock scripts. Anti-adblock filter lists currently rely on informal crowdsourced feedback from users to add/remove filter list rules. In this paper, we present the first comprehensive study of anti-adblock filter lists to analyze their effectiveness against anti-adblockers. Specifically, we compare and contrast the evolution of two popular anti-adblock filter lists. We show that these filter lists are implemented very differently even though they currently have a comparable number of filter list rules. We then use the Internet Archive's Wayback Machine to conduct a retrospective coverage analysis of these filter lists on Alexa top-5K websites over the span of last five years. We find that the coverage of these filter lists has considerably improved since 2014 and they detect anti-adblockers on about 9% of Alexa top-5K websites. To improve filter list coverage and speedup addition of new filter rules, we also design and implement a machine learning based method to automatically detect anti-adblock scripts using static JavaScript code analysis.

References

[1]

Acceptable ads program. https://adblockplus.org/acceptable-ads.

[2]

AdBlock. https://getadblock.com/.

[3]

AdBlock, Chrome web store. https://chrome.google.com/webstore/detail/adblock/gighmmpiobklfepjocnamgkkbiglidom?hl=en-US.

[4]

AdblockPlus. https://adblockplus.org/.

[5]

Adblock Plus, Chrome web store. https://chrome.google.com/webstore/detail/adblock-plus/cfhdojbkjhnklbpkdaibdccddilifddb.

[6]

Adblock Plus, Mozilla Firefox add-on. https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/.

[7]

Adblock rules list parser. https://github.com/shawa/adblockparser.

[8]

Anti-Adblock Killer. https://github.com/reek/anti-adblock-killer.

[9]

Anti-Adblock Killer List Forum. https://github.com/reek/anti-adblock-killer/issues.

[10]

BlockAdBlock. https://github.com/sitexw/BlockAdBlock/blob/master/blockadblock.js.

[11]

Blockzilla. https://zpacman.github.io/Blockzilla/.

[12]

Brave Browser. https://brave.com/.

[13]

Cliqz Browser. https://cliqz.com/us/.

[14]

Coalition for Better Ads. https://www.betterads.org/.

[15]

Disconnect.me filter list. https://disconnect.me/.

[16]

EasyList. https://easylist.to/.

[17]

EasyList Forum. https://forums.lanik.us.

[18]

EasyList variants. https://easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html.

[19]

Fanboy's Enhanced Tracking List. https://fanboy.co.nz/.

[20]

Filter lists syntax. https://adblockplus.org/en/filter-cheatsheet.

[21]

Firebug. http://getfirebug.com/.

[22]

Ghostery. https://www.ghostery.com/.

[23]

Ghostery, Chrome web store. https://chrome.google.com/webstore/detail/ghostery/mlomiejdfkolichcflejclcbmpeaniij?hl=en-US.

[24]

Ghostery, Mozilla Firefox add-on. https://addons.mozilla.org/en-US/firefox/addon/ghostery/.

[25]

HAR File. https://en.wikipedia.org/wiki/.har.

[26]

McAfee's URL categorization service. https://www.trustedsource.org/.

[27]

Mozilla Firefox. https://www.mozilla.org/en-US/firefox/.

[28]

Mozilla Firefox tracker blocking. https://testpilot.firefox.com/experiments/tracking-protection.

[29]

NetExport. https://getfirebug.com/wiki/index.php/Firebug_Extensions.

[30]

NoTrack Blocklist. https://github.com/quidsup/notrack.

[31]

PageFair, 2017 Global Adblock Report. https://pagefair.com/downloads/2017/01/PageFair-2017--Adblock-Report.pdf.

[32]

Privacy Badger. https://www.eff.org/privacybadger.

[33]

Privacy Badger, Chrome web store. https://chrome.google.com/webstore/detail/privacy-badger/pkehgijcmpdhfbdbbnkijodmdjhbjlgp?hl=en-US.

[34]

Privacy Badger, Mozilla Firefox add-on. https://addons.mozilla.org/en-US/firefox/addon/privacy-badger17/.

[35]

Selenium. http://docs.seleniumhq.org/.

[36]

Truth In Advertising, Federal Trade Commission. https://www.ftc.gov/news-events/media-resources/truth-advertising/.

[37]

Warning removal list. https://easylist-downloads.adblockplus.org/antiadblockfilters.txt.

[38]

Wayback Machine. https://archive.org/web/.

[39]

Wayback Machine API. https://archive.org/help/wayback_api.php.

[40]

Wayback Machine Archive Details. https://archive.org/about/.

[41]

YourAdChoices. http://youradchoices.com/.

[42]

A. Bosworth. A New Way to Control the Ads You See on Facebook, and an Update on Ad Blocking. https://newsroom.fb.com/news/2016/08/a-new-way-to-control-the-ads-you-see-on-facebook-and-an-update-on-ad-blocking/, 2016.

[43]

C. Curtsinger, B. Livshits, B. Zorn, and C. Seifert. ZOZZLE: Fast and Precise In-Browser JavaScript Malware Detection. In USENIX Security Symposium, 2011.

Digital Library

[44]

S. Englehardt and A. Narayanan. Online Tracking: A 1-million-site Measurement and Analysis. In ACM Conference on Computer and Communications Security (CCS), 2016.

Digital Library

[45]

S. Englehardt, D. Reisman, C. Eubank, P. Zimmerman, J. Mayer, A. Narayanan, and E. W. Felten. Cookies That Give You Away: The Surveillance Implications of Web Tracking. In World Wide Web (WWW) Conference, 2015.

Digital Library

[46]

Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. In Journal of Computer and System Sciences, 1997.

Digital Library

[47]

M. Graham. robots.txt meant for search engines don't work well for web archives. https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont- work- well- for-web-archives/, 2017.

[48]

D. Gugelmann, M. Happe, B. Ager, and V. Lenders. An Automated Approach for Complementing Ad Blockers' Blacklists. In Privacy Enhancing Technologies Symposium (PETS), 2015.

[49]

S. Ihm and V. S. Pai. Towards Understanding Modern Web Traffic. In ACM Internet Measurement Conference (IMC), 2011.

Digital Library

[50]

M. Ikram, H. J. Asghar, M. A. Kaafar, A. Mahanti, and B. Krishnamurthy. Towards Seamless Tracking-Free Web:Improved Detection of Trackers via One-class Learning. In Privacy Enhancing Technologies Symposium (PETS), 2017.

[51]

B. Krishnamurthy and C. E. Wills. Privacy Diffusion on the Web: A Longitudinal Perspective. In World Wide Web (WWW) Conference, 2009.

Digital Library

[52]

A. Lerner, T. Kohno, and F. Roesner. Rewriting History: Changing the Archived Web from the Present. In ACM Conference on Computer and Communications Security (CCS), 2017.

Digital Library

[53]

A. Lerner, A. K. Simpson, T. Kohno, and F. Roesner. Internet Jones and the Raiders of the Lost Trackers: An Archaeological Study of Web Tracking from 1996 to 2016. In USENIX Security Symposium, 2016.

Digital Library

[54]

X. Li, L. Wang, and E. Sung. AdaBoost with SVM-based component classifiers. In Engineering Applications of Artificial Intelligence, 2007.

Digital Library

[55]

M. Malloy, M. McNamara, A. Cahn, and P. Barford. Ad Blockers: Global Prevalence and Impact. In ACM Internet Measurement Conference (IMC), 2016.

Digital Library

[56]

J. Marshall. The Rise of the Anti-Ad Blockers. https://www.wsj.com/articles/the-rise-of-the-anti-ad-blockers-1465805039, 2016.

[57]

J. R. Mayer and J. C. Mitchell. Third-Party Web Tracking: Policy and Technology. In IEEE Symposium on Security and Privacy, 2012.

Digital Library

[58]

G. Merzdovnik, M. Huber, D. Buhov, N. Nikiforakis, S. Neuner, M. Schmiedecker, and E. Weippl. Block Me If You Can: A Large-Scale Study of Tracker-Blocking Tools. In IEEE European Symposium on Security and Privacy, 2017.

[59]

M. H. Mughees, Z. Qian, and Z. Shafiq. Detecting Anti Ad-blockers in the Wild. In Privacy Enhancing Technologies Symposium (PETS), 2017.

[60]

R. Nithyanand, S. Khattak, M. Javed, N. Vallina-Rodriguez, M. Falahrastegar, J. E. Powles, E. D. Cristofaro, H. Haddadi, and S. J. Murdoch. Adblocking and Counter-Blocking: A Slice of the Arms Race. In USENIX Workshop on Free and Open Communications on the Internet, 2016.

[61]

E. Pujol, O. Hohlfeld, and A. Feldmann. Annoyed Users: Ads and Ad-Block Usage in the Wild. In ACM Internet Measurement Conference (IMC), 2015.

Digital Library

[62]

M. Z. Rafique, T. V. Goethem, W. Joosen, C. Huygens, and N. Nikiforakis. It's Free for a Reason: Exploring the Ecosystem of Free Live Streaming Services. In Network and Distributed System Security Symposium (NDSS), 2016.

[63]

S. Ramaswamy. Building a better web for everyone. https://www.blog.google/topics/journalism-news/building-better-web-everyone/, 2017.

[64]

F. Roesner, T. Kohno, and D. Wetherall. Detecting and Defending Against Third-Party Tracking on the Web. In USENIX Symposium on Networked Systems Design and Implementation (NDSI), 2012.

Digital Library

[65]

G. Storey, D. Reisman, J. Mayer, and A. Narayanan. The Future of Ad Blocking: An Analytical Framework and New Techniques. In arXiv:1705.08568, 2017.

[66]

R. J. Walls, E. D. Kilmer, N. Lageman, and P. D. McDanie. Measuring the Impact and Perception of Acceptable Advertisements. In ACM Internet Measurement Conference (IMC), 2015.

Digital Library

[67]

J. Wilander. Apple Safari Intelligent Tracking Prevention. https://webkit.org/blog/7675/intelligent-tracking-prevention, 2017.

[68]

Y. Yang and J. O. Pedersen. A Comparative Study on Feature Selection in Text Categorization. In International Conference on Machine Learning, 1997.

Digital Library

[69]

Z. Yu, S. Macbeth, K. Modi, and J. M. Pujol. Tracking the Trackers. In World Wide Web (WWW) Conference, 2016.

Digital Library

Cited By

Amjad AMunir SShafiq ZGulzar MLuo BLiao XXu JKirda ELie D(2024)Blocking Tracking JavaScript at the Function GranularityProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670329(2177-2191)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670329
Roongta RGreenstadt RQuek TGao DZhou JCardenas A(2024)From User Insights to Actionable Metrics: A User-Focused Evaluation of Privacy-Preserving Browser ExtensionsProceedings of the 19th ACM Asia Conference on Computer and Communications Security10.1145/3634737.3657028(1753-1769)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3634737.3657028
Lee KLim CJin BKim TKim HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)AdFlush: A Real-World Deployable Machine Learning Solution for Effective Advertisement and Web Tracker PreventionProceedings of the ACM Web Conference 202410.1145/3589334.3645698(1902-1913)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645698
Show More Cited By

Index Terms

The ad wars: retrospective measurement and analysis of anti-adblock filter lists
1. Security and privacy
  1. Software and application security
    1. Web application security
  2. Systems security
    1. Browser security

Recommendations

ShadowBlock: A Lightweight and Stealthy Adblocking Browser
WWW '19: The World Wide Web Conference

As the popularity of adblocking has soared over the last few years, publishers are increasingly deploying anti-adblocking paywalls that ask users to either disable their adblockers or pay to access content. In this work we propose ShadowBlock, a new ...
Browser wars: Metaphor, Web browser, Microsoft, Internet Explorer, Netscape, Netscape Navigator, Mozilla Firefox, Google Chrome, Safari (web browser), Opera (web browser), Usage share of web browsers
AutoFR: Automated Filter Rule Generation for Adblocking
Adblocking relies on filter lists, which are manually curated and maintained by a community of filter list authors. Filter list curation is a laborious process that does not scale well to a large number of sites or over time. In this article, we introduce ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IMC '17: Proceedings of the 2017 Internet Measurement Conference

November 2017

509 pages

ISBN:9781450351188

DOI:10.1145/3131365

General Chairs:
Steve Uhlig
Queen Mary University of London
,
Olaf Maennel
Tallinn University of Technology

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

USENIX Assoc: USENIX Assoc

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IMC '17

Sponsor:

IMC '17: Internet Measurement Conference

November 1 - 3, 2017

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
728
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)8

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Amjad AMunir SShafiq ZGulzar MLuo BLiao XXu JKirda ELie D(2024)Blocking Tracking JavaScript at the Function GranularityProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670329(2177-2191)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670329
Roongta RGreenstadt RQuek TGao DZhou JCardenas A(2024)From User Insights to Actionable Metrics: A User-Focused Evaluation of Privacy-Preserving Browser ExtensionsProceedings of the 19th ACM Asia Conference on Computer and Communications Security10.1145/3634737.3657028(1753-1769)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3634737.3657028
Lee KLim CJin BKim TKim HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)AdFlush: A Real-World Deployable Machine Learning Solution for Effective Advertisement and Web Tracker PreventionProceedings of the ACM Web Conference 202410.1145/3589334.3645698(1902-1913)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645698
Lim KPark JKim DChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Phishing Vs. Legit: Comparative Analysis of Client-Side Resources of Phishing and Target Brand WebsitesProceedings of the ACM Web Conference 202410.1145/3589334.3645535(1756-1767)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645535
Hajj Chehade SSiby STroncoso C(2024)SINBAD: Saliency-informed detection of breakage caused by ad blocking2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00199(258-276)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00199
Wittig MKesdoğan D(2024)Detecting Web Tracking at the Network LayerICT Systems Security and Privacy Protection10.1007/978-3-031-56326-3_10(131-148)Online publication date: 24-Apr-2024
https://doi.org/10.1007/978-3-031-56326-3_10
Ahmad ZCasarin SCalzavara S(2023)An Empirical Analysis of Web Storage and Its Applications to Web TrackingACM Transactions on the Web10.1145/362338218:1(1-28)Online publication date: 11-Oct-2023
https://dl.acm.org/doi/10.1145/3623382
Heid KAndrae VHeider J(2023)Towards detecting device fingerprinting on iOS with API function hookingProceedings of the 2023 European Interdisciplinary Cybersecurity Conference10.1145/3590777.3590790(78-84)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3590777.3590790
Lee CSon SMeng WJensen CCremers CKirda E(2023)AdCPG: Classifying JavaScript Code Property Graphs with Explanations for Ad and Tracker BlockingProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623084(3505-3518)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3623084
Ghasemisharif MPolakis JMeng WJensen CCremers CKirda E(2023)Read Between the Lines: Detecting Tracking JavaScript with Bytecode ClassificationProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3616637(3475-3489)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3616637
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents