skip to main content
10.1145/3618257.3624803acmconferencesArticle/Chapter ViewAbstractPublication PagesimcConference Proceedingsconference-collections
research-article
Open access

Tracking, Profiling, and Ad Targeting in the Alexa Echo Smart Speaker Ecosystem

Published: 24 October 2023 Publication History

Abstract

Smart speakers collect voice commands, which can be used to infer sensitive information about users. Given the potential for privacy harms, there is a need for greater transparency and control over the data collected, used, and shared by smart speaker platforms as well as third party skills supported on them. To bridge this gap, we build a framework to measure data collection, usage, and sharing by the smart speaker platforms. We apply our framework to the Amazon smart speaker ecosystem. Our results show that Amazon and third parties, including advertising and tracking services that are unique to the smart speaker ecosystem, collect smart speaker interaction data. We also find that Amazon processes smart speaker interaction data to infer user interests and uses those inferences to serve targeted ads to users. Smart speaker interaction also leads to ad targeting and as much as 30X higher bids in ad auctions, from third party advertisers. Finally, we find that Amazon's and third party skills' data practices are often not clearly disclosed in their policy documents.

Supplemental Material

MP4 File
Presentation video for the IMC 2023 paper, entitled Tracking, Profiling, and Ad Targeting in the Alexa Echo Smart Speaker Ecosystem. The first half of the presentation highlights the privacy issues in smart speakers and discusses the challenges in auditing smart speaker platforms and the third party skills(apps) running on them. The second half of the presentation describes the details of our proposed framework that can audit smart speakers without the cooperation of the smart speaker platform. The third half of the presentation describes our findings, which uncovers the online services that collect user data, describes how user data is used for non-essential purposes (such as online targeted advertising), and whether the third party skills and the platform vendor disclose their practices in their policies.

References

[1]
Aaron M Spelling. 2022. Dating and Relationship Tips and advices. https://www.amazon.com/dp/B07YCKFCCF.
[2]
Amazon.com, Inc. 2022a. Alexa and Alexa Device FAQs. https://www.amazon.com/gp/help/customer/display.html?nodeId=201602230.
[3]
Amazon.com, Inc. 2022b. Alexa and Alexa Device FAQs (archived September 2022). https://web.archive.org/web/20220901073936/http://www.amazon.com/gp/help/customer/display.html?nodeId=201602230.
[4]
Amazon.com, Inc. 2022c. Alexa Blogs: Advertising and Alexa. https://developer.amazon.com/blogs/alexa/post/54c3a0f8-5b29-4071-acd7-2b832b860c83/advertising-and-alexa.
[5]
Amazon.com, Inc. 2022d. Alexa-hosted Skills. https://developer.amazon.com/en-US/docs/alexa/hosted-skills/build-a-skill-end-to-end-using-an-alexa-hosted-skill.html.
[6]
Amazon.com, Inc. 2022 e. Alexa Privacy Hub. https://www.amazon.com/Alexa-Privacy-Hub/b?ie=UTF8&node=19149155011.
[7]
Amazon.com, Inc. 2022 f. Alexa Privacy Hub (archived October 2022). https://web.archive.org/web/20221010091208/http://www.amazon.com/b/?node=19149155011.
[8]
Amazon.com, Inc. 2022 g. Alexa Skill Certification Requirements. https://developer.amazon.com/en-US/docs/alexa/custom-skills/certification-requirements-for-custom-skills.html.
[9]
Amazon.com, Inc. 2022 h. Alexa Skills Policy Testing. https://developer.amazon.com/en-US/docs/alexa/custom-skills/policy-testing-for-an-alexa-skill.html.
[10]
Amazon.com, Inc. 2022 i. Alexa Skills Privacy Requirements. https://developer.amazon.com/en-US/docs/alexa/custom-skills/security-testing-for-an-alexa-skill.html#25-privacy-requirements.
[11]
Amazon.com, Inc. 2022 j. Amazon: Request Your Data. https://www.amazon.com/gp/privacycentral/dsar/preview.html.
[12]
Amazon.com, Inc. 2022 k. Amazon.com Privacy Notice. https://www.amazon.com/gp/help/customer/display.html?nodeId=GX7NJQ4ZB8MHFRNJ.
[13]
Amazon.com, Inc. 2022 l. Audio Ads - Create audio advertising campaigns. https://advertising.amazon.com/en-ca/solutions/products/audio-ads.
[14]
Amazon.com, Inc. 2022 m. AVS Testing and Certification Process. https://developer.amazon.com/en-US/docs/alexa/alexa-voice-service/product-testing-overview.html.
[15]
Amazon.com, Inc. 2022 n. Configure Permissions for Customer Information in Your Skill. https://developer.amazon.com/en-US/docs/alexa/custom-skills/configure-permissions-for-customer-information-in-your-skill.html.
[16]
Amazon.com, Inc. 2022 o. Managing advertising preferences on Alexa. https://www.amazon.com/b/?node=98592480011.
[17]
Amazon.com, Inc. 2022 p. Module 2: Design an Engaging Voice User Interface. https://developer.amazon.com/en-US/alexa/alexa-skills-kit/get-deeper/tutorials-code-samples/build-an-engaging-alexa-skill/module-2.
[18]
Amazon.com, Inc. 2022 q. Policy Testing. https://developer.amazon.com/en-US/docs/alexa/custom-skills/policy-testing-for-an-alexa-skill.html#advertising.
[19]
Benjamin Andow, Samin Yaseer Mahmud, Justin Whitaker, William Enck, Bradley Reaves, Kapil Singh, and Serge Egelman. 2020. Actions Speak Louder than Words: Entity-Sensitive Privacy Policy and Data Flow Analysis with PoliCheck. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, Boston, 985--1002. https://www.usenix.org/conference/usenixsecurity20/presentation/andow
[20]
Rubén Barceló-Armada, Ismael Castell-Uroz, and Pere Barlet-Ros. 2022. Amazon Alexa traffic traces. Computer Networks, Vol. 205 (2022), 108782.
[21]
Muhammad Ahmad Bashir, Sajjad Arshad, William Robertson, and Christo Wilson. 2016. Tracing Information Flows Between Ad Exchanges Using Retargeted Ads. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, 481--496.
[22]
Long Cheng, Christin Wilson, Song Liao, Jeffrey Young, Daniel Dong, and Hongxin Hu. 2020. Dangerous Skills Got Certified: Measuring the Trustworthiness of Skill Certification in Voice Personal Assistant Platforms. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (CCS '20). Association for Computing Machinery, Virtual, 1699--1716.
[23]
Thomas Claburn. 2022. Study: How Amazon uses Echo smart speaker conversations to target ads. https://theregister.com/2022/04/27/amazon_audio_data/.
[24]
John Cook, Rishab Nithyanand, and Zubair Shafiq. 2020. Inferring Tracker-Advertiser Relationships in the Online Advertising Ecosystem using Header Bidding. In Proceedings on Privacy Enhancing Technologies, Vol. 2020 (1). Sciendo, Virtual, 65--82.
[25]
Crunchbase Inc. 2022. Crunchbase. https://www.crunchbase.com/.
[26]
Daniel J Dubois, Roman Kolcun, Anna Maria Mandalari, Muhammad Talha Paracha, David Choffnes, and Hamed Haddadi. 2020. When Speakers Are All Ears: Characterizing Misactivations of IoT Smart Speakers. In Proceedings on Privacy Enhancing Technologies, Vol. 2020 (4). Sciendo, Virtual, 255--276.
[27]
DuckDuckGo. 2022. Tracker Radar (list of entities). https://github.com/duckduckgo/tracker-radar/tree/main/entities.
[28]
Pierre N. Durette. 2022. gTTS (Google Text-to-Speech), a Python library and CLI tool to interface with Google Translate text-to-speech API. https://pypi.org/project/gTTS/.
[29]
Dyson Limited. 2022. Dyson. https://www.amazon.com/dp/B06WVN7SHC.
[30]
Jide S. Edu, Xavier Ferrer-Aran, Jose Such, and Guillermo Suarez-Tangil. 2023. SkillVet: Automated Traceability Analysis of Amazon Alexa Skills. IEEE Transactions on Dependable and Secure Computing, Vol. 20, 1 (2023), 161--175.
[31]
Steven Englehardt and Arvind Narayanan. 2016. Online Tracking: A 1-Million-Site Measurement and Analysis. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). Association for Computing Machinery, Vienna, 1388--1401.
[32]
Geoffrey A. Fowler. 2019. Alexa has been eavesdropping on you this whole time. https://www.washingtonpost.com/technology/2019/05/06/alexa-has-been-eavesdropping-you-this-whole-time/.
[33]
Garmin International. 2022. Garmin. https://www.amazon.com/dp/B075TRB4V5.
[34]
Gary Horcher. 2018. Woman says her Amazon device recorded private conversation, sent it out to random contact. https://www.kiro7.com/news/local/woman-says-her-amazon-device-recorded-private-conversation-sent-it-out-to-random-contact/755507974/.
[35]
Genesis Motors USA. 2022. Genesis. https://www.amazon.com/dp/B01JXP09PI.
[36]
Aniketh Girish, Tianrui Hu, Vijay Prakash, Daniel J. Dubois, Srdjan Matic, Danny Yuxing, Serge Egelman, Joel Reardon, Juan Tapiador, David Choffnes, and Narseo Vallina-Rodriguez. 2023. In the Room Where It Happens: Characterizing Local Communication and Threats in Smart Homes. In Proceedings of the ACM Internet Measurement Conference (IMC '23). Association for Computing Machinery, Montréal.
[37]
Google, Inc. 2022a. Header Bidding. https://admanager.google.com/home/resources/feature-brief-open-bidding/.
[38]
Google, Inc. 2022b. Real-time Bidding. https://developers.google.com/authorized-buyers/rtb/start.
[39]
Google, Inc. 2022c. RTB - Cookie Matching. https://developers.google.com/authorized-buyers/rtb/cookie-guide.
[40]
Tianrui Hu, Daniel J. Dubois, and David Choffnes. 2023. BehavIoT: Measuring Smart Home IoT Behavior Using Network-Inferred Behavior Models. In Proceedings of the ACM Internet Measurement Conference (IMC '23). Association for Computing Machinery, Montréal.
[41]
Danny Yuxing Huang, Noah Apthorpe, Frank Li, Gunes Acar, and Nick Feamster. 2020. IoT Inspector: Crowdsourcing Labeled Network Traffic from Smart Home Devices at Scale. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 4. Association for Computing Machinery, New York, NY, USA, Article 46, 21 pages. https://doi.org/10.1145/3397333
[42]
ICM. 2022. Air Quality Report. https://www.amazon.com/dp/B01EOFCHMA.
[43]
In Touch Ministries. 2022. Charles Stanley Radio. https://www.amazon.com/dp/B07FF2QGXW.
[44]
Umar Iqbal. 2022. Your Echos are Heard: Tracking, Profiling, and Ad Targeting in the Amazon Smart Speaker Ecosystem, FTC PrivacyCon 2022. https://www.ftc.gov/news-events/events/2022/11/privacycon-2022.
[45]
Umar Iqbal, Charlie Wolfe, Charles Nguyen, Steven Englehardt, and Zubair Shafiq. 2022. Khaleesi: Breaker of Advertising and Tracking Request Chains. In 31st USENIX Security Symposium. USENIX Association, Boston, 2911--2928.
[46]
iRobot. 2022. iRobot Home. https://www.amazon.com/dp/B06Y3PSHQ3.
[47]
Huafeng Jin and Shuo Wang. 2018. Voice-based determination of physical and emotional characteristics of users. US Patent 10096319B1.
[48]
Kevel. 2022. Header Bidding (HBIX) 2021 Tracker. https://www.kevel.co/hbix/.
[49]
Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2019. Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation. In 26th Annual Network and Distributed System Security Symposium. Internet Society, San Diego.
[50]
Christopher Lentzsch, Sheel Jayesh Shah, Benjamin Andow, Martin Degeling, Anupam Das, and William Enck. 2021. Hey Alexa, is this skill safe?: Taking a closer look at the Alexa skill ecosystem. In 28th Annual Network and Distributed System Security Symposium. The Internet Society, San Diego.
[51]
Logitech. 2022. Harmony. https://www.amazon.com/dp/B01M4LDPX3.
[52]
Sapna Maheshwari. 2018. Hey, Alexa, What Can You Hear? And What Will You Do With It? https://www.nytimes.com/2018/03/31/business/media/amazon-google-privacy-digital-assistants.html.
[53]
Anna Maria Mandalari, Daniel J Dubois, Roman Kolcun, Muhammad Talha Paracha, Hamed Haddadi, and David Choffnes. 2021. Blocking Without Breaking: Identification and Mitigation of Non-Essential IoT Traffic. In Proceedings on Privacy Enhancing Technologies, Vol. 2021 (4). Sciendo, Virtual, 369--388.
[54]
M. Hammad Mazhar and Zubair Shafiq. 2020. Characterizing Smart Home IoT Traffic in the Wild. In 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Implementation (IoTDI). IEEE, Sydney, 203--215.
[55]
Men's Finest. 2022. Men's Finest Daily Fashion Tip. https://www.amazon.com/dp/B07CB3ZN6N.
[56]
Hooman Mohajeri Moghaddam. 2022. Tracking and Behavioral Targeting on Connected TV Platforms. (2022). https://dataspace.princeton.edu/handle/88435/dsp010p096b14c Doctoral Disseration.
[57]
Hooman Mohajeri Moghaddam, Gunes Acar, Ben Burgess, Arunesh Mathur, Danny Yuxing Huang, Nick Feamster, Edward W. Felten, Prateek Mittal, and Arvind Narayanan. 2019. Watching You Watch: The Tracking Ecosystem of Over-the-Top TV Streaming Devices. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS '19). Association for Computing Machinery, London, 131--147.
[58]
Tom Nardi. 2019. Uncovering the Echo Dot's Hidden USB Port. https://hackaday.com/2019/08/15/uncovering-the-echo-dots-hidden-usb-port/.
[59]
Lukasz Olejnik, Minh-Dung Tran, and Claude Castelluccia. 2014. Selling Off Privacy at Auction. In 21st Annual Network and Distributed System Security Symposium. The Internet Society, San Diego.
[60]
Panagiotis Papadopoulos, Nicolas Kourtellis, Pablo Rodriguez Rodriguez, and Nikolaos Laoutaris. 2017. If You Are Not Paying for It, You Are the Product: How Much Do Advertisers Pay to Reach You?. In Proceedings of the 2017 Internet Measurement Conference. Association for Computing Machinery, London, 142--156.
[61]
Picovoice Inc. 2022. Porcupine Wake Word Detection & Keyword Spotting. https://picovoice.ai/platform/porcupine/.
[62]
Prebid.org Inc. 2022. Prebid. https://prebid.org/.
[63]
Raspberry Pi. 2021. Setting up a Bridged Wireless Access Point. https://github.com/raspberrypi/documentation/blob/develop/documentation/asciidoc/computers/configuration/access-point-bridged.adoc.
[64]
Jingjing Ren, Daniel J. Dubois, David Choffnes, Anna Maria Mandalari, Roman Kolcun, and Hamed Haddadi. 2019. Information Exposure From Consumer IoT Devices: A Multidimensional, Network-Informed Measurement Approach. In Proceedings of the Internet Measurement Conference (IMC '19). Association for Computing Machinery, Amsterdam, 267--279.
[65]
Rhasspy. 2022. Rhasspy Voice Assistant. https://rhasspy.readthedocs.io/.
[66]
Said Jawad Saidi, Anna Maria Mandalari, Roman Kolcun, Hamed Haddadi, Daniel J. Dubois, David Choffnes, Georgios Smaragdakis, and Anja Feldmann. 2020. A Haystack Full of Needles: Scalable Detection of IoT Devices in the Wild. In Proceedings of the ACM Internet Measurement Conference (IMC '20). Association for Computing Machinery, Pittsburgh, 87--100.
[67]
Hamza Shaban. 2018. Amazon Alexa user receives 1,700 audio recordings of a stranger through `human error'. https://www.washingtonpost.com/technology/2018/12/20/amazon-alexa-user-receives-audio-recordings-stranger-through-human-error/.
[68]
Rita Singh. 2019. Profiling humans from their voice. Springer.
[69]
Software Freedom Conservancy. 2022. Selenium. https://www.selenium.dev/.
[70]
Sonos, Inc. 2022. Sonos. https://www.amazon.com/dp/B072ML3N6K.
[71]
Statista. 2022. Number of households with smart home products and services in use worldwide from 2015 to 2025. https://www.statista.com/statistics/1252975/smart-home-households-worldwide/.
[72]
Statista. 2022. Smart speaker devices installed base in the United States from 2017 to 2020. https://www.statista.com/statistics/794480/us-amazon-echo-google-home-installed-base/.
[73]
Rahmadi Trimananda, Hieu Le, Hao Cui, Janice Tran Ho, Anastasia Shuba, and Athina Markopoulou. 2022. OVRseen: Auditing Network Traffic and Privacy Policies in Oculus VR. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, 3789--3806.
[74]
ttm. 2022. Essential Oil Benefits. https://www.amazon.com/dp/B074CNX3G9.
[75]
Janus Varmarken, Hieu Le, Anastasia Shuba, Athina Markopoulou, and Zubair Shafiq. 2020. The TV is Smart and Full of Trackers: Measuring Smart TV Advertising and Tracking. In Proceedings on Privacy Enhancing Technologies, Vol. 2020 (2). Sciendo, Virtual, 129--154.
[76]
VCA, INC. 2022. VCA Animal Hospital. https://amazon.com/dp/B07KYS1Y1X.
[77]
WaLLy3K. 2022. Pi-hole Blocklist. https://firebog.net/.
[78]
Max Willens. 2021. Amid post-cookie confusion, Amazon plans to launch an identifier of its own. https://digiday.com/marketing/amid-post-cookie-confusion-amazon-explores-launching-an-identifier-of-its-own/amp/.
[79]
Xeline Development. 2022. Makeup of the Day. https://amazon.com/dp/B072N6BNB1.
[80]
Jeffrey Young, Song Liao, Long Cheng, Hongxin Hu, and Huixing Deng. 2022. SkillDetective: Automated Policy-Violation Detection of Voice Assistant Applications in the Wild. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, 1113--1130.
[81]
YouVersion. 2022. YouVersion Bible. https://www.amazon.com/dp/B017RXFNKY.
[82]
Eric Zeng, Tadayoshi Kohno, and Franziska Roesner. 2021. What Makes a ?Bad" Ad? User Perceptions of Problematic Online Advertising. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 361, 24 pages. https://doi.org/10.1145/3411764.3445459
[83]
Jiang Zhang, Konstantinos Psounis, Muhammad Haroon, and Zubair Shafiq. 2022. HARPO: Learning to Subvert Online Behavioral Advertising. In 29th Annual Network and Distributed System Security Symposium. Internet Society, San Diego.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IMC '23: Proceedings of the 2023 ACM on Internet Measurement Conference
October 2023
746 pages
ISBN:9798400703829
DOI:10.1145/3618257
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2023

Check for updates

Badges

  • Best Paper

Author Tags

  1. alexa
  2. amazon
  3. corporate surveillance
  4. data collection
  5. echo
  6. smart speaker
  7. targeted advertising

Qualifiers

  • Research-article

Funding Sources

Conference

IMC '23
Sponsor:
IMC '23: ACM Internet Measurement Conference
October 24 - 26, 2023
Montreal QC, Canada

Acceptance Rates

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,218
  • Downloads (Last 6 weeks)165
Reflects downloads up to 04 Nov 2024

Other Metrics

Citations

Cited By

View all

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media