note

Public Access

Enhancing Lifelogging Privacy by Detecting Screens

Authors:

Mohammed Korayem,

Robert Templeman,

David Crandall,

Apu KapadiaAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 4309 - 4314

https://doi.org/10.1145/2858036.2858417

Published: 07 May 2016 Publication History

Abstract

Low-cost, lightweight wearable cameras let us record (or 'lifelog') our lives from a 'first-person' perspective for purposes ranging from fun to therapy. But they also capture private information that people may not want to be recorded, especially if images are stored in the cloud or visible to other people. For example, recent studies suggest that computer screens may be lifeloggers' single greatest privacy concern, because many people spend a considerable amount of time in front of devices that display private information. In this paper, we investigate using computer vision to automatically detect computer screens in photo lifelogs. We evaluate our approach on an existing in-situ dataset of 36 people who wore cameras for a week, and show that our technique could help manage privacy in the upcoming era of wearable cameras.

References

[1]

Apple. Siri. https://www.apple.com/ios/siri/.

[2]

Autographer. Autographer Wearable Camera. http://autographer.com.

[3]

Jeff Brown. ZBar bar code reader. http://zbar.sourceforge.net/index.html.

[4]

K. E. Caine. 2009. Supporting privacy by preventing misclosure. In CHI'09 Extended Abstracts on Human Factors in Computing Systems. ACM, 3145-3148.

Digital Library

[5]

Kelly E. Caine, Celine Y. Zimmerman, Zachary Schall-Zimmerman, William R. Hazlewood, Alexander C. Sulgrove, L. Jean Camp, Katherine H. Connelly, Lesa L. Huber, and Kalpana Shankar. 2010. DigiSwitch: Design and Evaluation of a Device for Older Adults to Preserve Privacy While Monitoring Health at Home. In Proceedings of the 1st ACM International Health Informatics Symposium. 153-162.

Digital Library

[6]

D. Castro, S. Hickson, V. Bettadapura, E. Thomaz, G. Abowd, H. Christensen, and I. Essa. 2015. Predicting daily activities from egocentric images using deep learning. In Proceedings of the ACM International Symposium on Wearable Computers.

Digital Library

[7]

Ken Chatfield, Victor Lempitsky, Andrea Vedaldi, and Andrew Zisserman. 2011. The devil is in the details: an evaluation of recent feature encoding methods. In Proceedings of the British Machine Vision Conference.

[8]

Fanny Coudert, Denis Butin, and Daniel Le Metayer. 2015. Body-worn cameras for police accountability: Opportunities and risks. Computer Law & Security Review 31 (2015), 749-762.

[9]

Navneet Dalal and Bill Triggs. 2005. Histograms of Oriented Gradients for Human Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 886-893.

Digital Library

[10]

C. Delo. 2013. U.S. Adults Now Spending More Time on Digital Devices Than Watching TV. Advertising Age (2013).

[11]

T. Denning, Z. Dehlawi, and T. Kohno. 2014. In Situ with Bystanders of Augmented Reality Glasses: Perspectives on Recording and Privacy-Mediating Technologies. In Proceedings of the International Conference on Human Factors in Computing Systems.

Digital Library

[12]

K. Fukuchi. libqrencode. http://http://fukuchi.org/works/qrencode/.

[13]

Google. Google Glass. https://plus.google.com/+GoogleGlass/posts.

[14]

Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi, James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur, and Ken Wood. 2006. SenseCam: a Retrospective Memory Aid. In Proceedings of the ACM International Conference on Ubiquitous Computing.

Digital Library

[15]

Roberto Hoyle, Robert Templeman, Denise Anthony, David Crandall, and Apu Kapadia. 2015. Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras. In ACM SIGCHI Conference on Human Factors in Computing Systems.

Digital Library

[16]

R. Hoyle, R. Templeman, S. Armes, D. Anthony, D. Crandall, and A. Kapadia. 2014. Privacy behaviors of lifeloggers using wearable cameras. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 571-582.

Digital Library

[17]

S. Jana, D. Molnar, A. Moshchuk, A. M. Dunn, B. Livshits, H. J. Wang, and E. Ofek. 2013a. Enabling Fine-Grained Permissions for Augmented Reality Applications with Recognizers. In USENIX Security. 415-430.

Digital Library

[18]

S. Jana, A. Narayanan, and V. Shmatikov. 2013b. A Scanner Darkly: Protecting User Privacy From Perceptual Applications. In Proceedings of the IEEE Symposium on Security and Privacy.

Digital Library

[19]

Yangqing Jia. 2013. Caffe: An Open Source Convolutional Architecture for Fast Feature Embedding. http://caffe.berkeleyvision.org/. (2013).

[20]

Apu Kapadia, Tristan Henderson, Jeffrey J. Fielding, and David Kotz. 2007. Virtual Walls: Protecting Digital Privacy in Pervasive Environments. In Proceedings of the Fifth International Conference on Pervasive Computing, Vol. 4480. 162-179.

Digital Library

[21]

P. Klemperer, Y. Liang, M. Mazurek, M. Sleeper, B. Ur, L. Bauer, L.F. Cranor, N. Gupta, and M. Reiter. 2012. Tag, you can see it!: using tags for access control in photo sharing. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, 377-386.

Digital Library

[22]

Mohammed Korayem, Robert Templeman, Dennis Chen, David Crandall, and Apu Kapadia. 2014. ScreenAvoider: Protecting Computer Screens from Ubiquitous Cameras. arXiv 1412.0008.

[23]

A. Krizhevsky, I. Sutskever, and G.E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097-1105.

[24]

Michael S. Lam, Suneeta Godbole, Jacqueline Chen, Melody Oliver, Hannah Badland, Simon J. Marshall, Paul Kelly, Charlie Foster, Aiden Doherty, and Jacqueline Kerr. 2013. Measuring Time Spent Outdoors Using a Wearable Camera and GPS. In Proceedings of the 4th International SenseCam Pervasive Imaging Conference. 1-7.

Digital Library

[25]

Marc Langheinrich. 2002. A Privacy Awareness System for Ubiquitous Computing Environments. In Proceedings of the ACM International Conference on Ubiquitous Computing. Vol. 2498. 237-245.

Digital Library

[26]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 11, 86 (1998), 2278-2324.

[27]

David G. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 2 (Nov. 2004), 91-110.

Digital Library

[28]

Gabriela Marcu, Anind K. Dey, and Sara Kiesler. 2012. Parent-driven Use of Wearable Cameras for Autism Support: A Field Study with Families. In Proceedings of the ACM International Conference on Ubiquitous Computing. 401-410.

Digital Library

[29]

Narrative. Narrative Clip: a wearable, automatic lifelogging camera. http://getnarrative.com.

[30]

M. Oquab, L. Bottou, I. Laptev, and J. Sivic. 2014. Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[31]

Nisarg Raval, Landon Cox, Animesh Srivastava, Ashwin Machanavajjhala, and Kiron Lebeck. 2014. MarkIt: privacy markers for protecting visual secrets. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. 1289-1295.

Digital Library

[32]

Franziska Roesner, Tadayoshi Kohno, and David Molnar. 2013. Security and Privacy for Augmented Reality Systems. Commun. ACM (2013).

Digital Library

[33]

Franziska Roesner, David Molnar, Alexander Moshchuk, Tadayoshi Kohno, and Helen J Wang. 2014. World-Driven Access Control for Continuous Sensing. Microsoft Tech Report (2014).

[34]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2014. ImageNet Large Scale Visual Recognition Challenge. arXiv 1409.0575.

[35]

R. Templeman, A. Kapadia, R. Hoyle, and D. Crandall. 2014a. Reactive Security: Responding to Visual Stimuli from Wearable Cameras. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. 1297-1306.

Digital Library

[36]

R Templeman, M. Korayem, D. Crandall, and K. Kapadia. 2014b. PlaceAvoider: Steering First-Person Cameras away from Sensitive Spaces. In Network and Distributed System Security Symposium.

[37]

Edison Thomaz, Aman Parnami, Jonathan Bidwell, Irfan Essa, and Gregory D. Abowd. 2013. Technological Approaches for Addressing Privacy Concerns when Recognizing Eating Behaviors with Wearable Cameras. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 739-748.

Digital Library

[38]

Nancy A Van House and Marc Davis. 2005. The social life of cameraphone images. In Proceedings of the Pervasive Image Capture and Sharing: New Social Practices and Implications for Technology Workshop at the Seventh International Conference on Ubiquitous Computing.

[39]

Jing Wang, Grant Schindler, and Irfan Essa. 2012. Orientation-aware Scene Understanding for Mobile Cameras. In Proceedings of the ACM International Conference on Ubiquitous Computing. 260-269.

Digital Library

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/3672076Online publication date: 14-Jun-2024
https://doi.org/10.1145/3672076
Xu AZhou ZMiyazaki KYoshikawa RHosio SYatani K(2024)DIPA2Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314397:4(1-30)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631439
Plizzari CGoletto GFurnari ABansal SRagusa FFarinella GDamen DTommasi T(2024)An Outlook into the Future of Egocentric VisionInternational Journal of Computer Vision10.1007/s11263-024-02095-7132:11(4880-4936)Online publication date: 28-May-2024
https://doi.org/10.1007/s11263-024-02095-7
Show More Cited By

Index Terms

Enhancing Lifelogging Privacy by Detecting Screens
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
2. Social and professional topics
  1. Computing / technology policy

Recommendations

Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

While media reports about wearable cameras have focused on the privacy concerns of bystanders, the perspectives of the `lifeloggers' themselves have not been adequately studied. We report on additional analysis of our previous in-situ lifelogging study ...
Privacy behaviors of lifeloggers using wearable cameras
UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing

A number of wearable 'lifelogging' camera devices have been released recently, allowing consumers to capture images and other sensor data continuously from a first-person perspective. Unlike traditional cameras that are used deliberately and ...
Understanding lifelog sharing preferences of lifeloggers
OzCHI '16: Proceedings of the 28th Australian Conference on Computer-Human Interaction

The lifelogging activity enables users, the lifeloggers, to passively capture images using wearable cameras from a first person perspective and ultimately create a visual diary encoding every possible aspect of their life with unprecedented details. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

© 2016 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Note

Funding Sources

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
1,309
Total Downloads

Downloads (Last 12 months)193
Downloads (Last 6 weeks)22

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/3672076Online publication date: 14-Jun-2024
https://doi.org/10.1145/3672076
Xu AZhou ZMiyazaki KYoshikawa RHosio SYatani K(2024)DIPA2Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314397:4(1-30)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631439
Plizzari CGoletto GFurnari ABansal SRagusa FFarinella GDamen DTommasi T(2024)An Outlook into the Future of Egocentric VisionInternational Journal of Computer Vision10.1007/s11263-024-02095-7132:11(4880-4936)Online publication date: 28-May-2024
https://doi.org/10.1007/s11263-024-02095-7
Bracamonte VPape SLöbner S(2024)Factors of Intention to Use a Photo Tool: Comparison Between Privacy-Enhancing and Non-privacy-enhancing ToolsICT Systems Security and Privacy Protection10.1007/978-3-031-56326-3_23(321-334)Online publication date: 24-Apr-2024
https://doi.org/10.1007/978-3-031-56326-3_23
Xu AZhou ZMiyazaki KYoshikawa RHosio SYatani K(2023)DIPA : An Image Dataset with Cross-cultural Privacy Concern AnnotationsCompanion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584176(259-266)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581754.3584176
Elagroudy PKhamis MMathis FIrmscher DSood EBulling ASchmidt A(2023)Impact of Privacy Protection Methods of Lifelogs on Remembered MemoriesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581565(1-10)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581565
Tabet SKayssi AElhajj I(2023)Mobile Diminished Reality for Preserving 3D Visual Privacy2023 International Conference on Intelligent Metaverse Technologies & Applications (iMETA)10.1109/iMETA59369.2023.10294436(01-07)Online publication date: 18-Sep-2023
https://doi.org/10.1109/iMETA59369.2023.10294436
Li YCaine K(2022)Obfuscation Remedies Harms Arising from Content Flagging of PhotosProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517520(1-25)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517520
Engelsma JJain ABoddeti V(2022)HERS: Homomorphically Encrypted Representation SearchIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2021.31398664:3(349-360)Online publication date: Jul-2022
https://doi.org/10.1109/TBIOM.2021.3139866
Shahi SAlharbi RGao YSen SKatsaggelos AHester JAlshurafa N(2022)Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767447(341-346)Online publication date: 21-Mar-2022
https://doi.org/10.1109/PerComWorkshops53856.2022.9767447
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten