research-article

Public Access

Characterizing the Reconfiguration Latency of Image Sensor Resolution on Android Devices

Authors:

Vraj Delhivala,

Robert LiKamWaAuthors Info & Claims

HotMobile '18: Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications

Pages 81 - 86

https://doi.org/10.1145/3177102.3177109

Published: 12 February 2018 Publication History

All formats PDF

Abstract

Advances in vision processing have ignited a proliferation of mobile vision applications, including augmented reality. However, limited by the inability to rapidly reconfigure sensor operation for performance-efficiency tradeoffs, high power consumption causes vision applications to drain the device's battery. To explore the potential impact of enabling rapid reconfiguration, we use a case study around marker-based pose estimation to understand the relationship between image frame resolution, task accuracy, and energy efficiency. Our case study motivates that to balance energy efficiency and task accuracy, the application needs to dynamically and frequently reconfigure sensor resolution.

To explore the latency bottlenecks to sensor resolution reconfiguration, we define and profile the end-to-end reconfiguration latency and frame-to-frame latency of changing capture resolution on a Google LG Nexus 5X device. We identify three major sources of sensor resolution reconfiguration latency in current Android systems: (i) sequential configuration patterns, (ii) expensive system calls, and (iii) imaging pipeline delay. Based on our intuitions, we propose a redesign of the Android camera system to mitigate the sources of latency. Enabling smooth transitions between sensor configurations will unlock new classes of adaptive-resolution vision applications.

References

[1]

Andrew Adams, Eino-Ville Talvala, Sung Hee Park, David E. Jacobs, Boris Ajdin, Natasha Gelfand, Jennifer Dolson, Daniel Vaquero, Jongmin Baek, Marius Tico, Hendrik P. A. Lensch, Wojciech Matusik, Kari Pulli, Mark Horowitz, and Marc Levoy 2010. The Frankencamera: An Experimental Platform for Computational Photography Proceedings of the 37th International Conference and Exhibition on Computer Graphics and Interactive Techniques.

Digital Library

[2]

Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report. Carnegie Mellon University-CS-16--118, Carnegie Mellon University School of Computer Science.

[3]

Mark Buckler, Suren Jayasuriya, and Adrian Sampson. 2017. Reconfiguring the Imaging Pipeline for Computer Vision. CoRR (2017).

[4]

Tiffany Yu-Han Chen, Lenin S. Ravindranath, Shuo Deng, Paramvir Victor Bahl, and Hari Balakrishnan 2015. Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems.

Digital Library

[5]

Sony Semiconductor Solutions Corporation. 2017. IMX377CQT ProductSummary v1.5. (2017).

[6]

KHRONOS Group. 2013. Camera BOF. (2013). https://www.khronos.org/assets/uploads/developers/library/2013-siggraph-camera-bof/Camera-BOF_SIGGRAPH-2013.pdf

[7]

Tomas Hruby, Teodor Crivat, Herbert Bos, and Andrew S. Tanenbaum 2014. On Sockets and System Calls Minimizing Context Switches for the Socket API Proceedings of the 2014 International Conference on Timely Results in Operating Systems.

Digital Library

[8]

Google Inc. 2017. Android Developers: CaptureResult. (2017). https://developer.android.com/reference/android/hardware/camera2/CaptureResult.html

[9]

Seungwoo Kang, Jinwon Lee, Hyukjae Jang, Hyonik Lee, Youngki Lee, Souneil Park, Taiwoo Park, and Junehwa Song. 2008. SeeMon: Scalable and Energy-efficient Context Monitoring Framework for Sensor-rich Mobile Environments. In Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services.

Digital Library

[10]

L. Li, J. Wang, X. Wang, H. Ye, and Z. Hu 2017. SceneMan: Bridging mobile apps with system energy manager via scenario notification Proceedings of the 2017 IEEE/ACM International Symposium on Low Power Electronics and Design.

[11]

Robert LiKamWa, Yunhui Hou, Julian Gao, Mia Polansky, and Lin Zhong 2016. RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision Proceedings of the 43rd International Symposium on Computer Architecture.

Digital Library

[12]

Robert LiKamWa, Bodhi Priyantha, Matthai Philipose, Lin Zhong, and Paramvir Bahl 2013. Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Services.

Digital Library

[13]

Mahadev Satyanarayanan. 2015. A Brief History of Cloud Offload: A Personal Journey from Odyssey Through Cyber Foraging to Cloudlets. GetMobile: Mobile Comp. and Comm. Vol. 18 (2015).

Digital Library

[14]

Livio Soares and Michael Stumm 2010. FlexSC: Flexible System Call Scheduling with Exception-less System Calls Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation.

Digital Library

[15]

Open Source Computer Vision. 2016. Real Time pose estimation of a textured object. (2016). http://docs.opencv.org/3.2.0/dc/d2c/tutorial_real_time_pose.html

[16]

Yin Yan, Shaun Cosgrove, Varun Anand, Amit Kulkarni, Sree Harsha Konduri, Steven Y. Ko, and Lukasz Ziarek 2014. Real-time Android with RTDroid. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services.

Digital Library

Cited By

Hu JShaikh ABahremand ALiKamWa R(2021)Characterizing real-time dense point cloud capture and streaming on mobile devicesProceedings of the 3rd ACM Workshop on Hot Topics in Video Analytics and Intelligent Edges10.1145/3477083.3480155(1-6)Online publication date: 25-Oct-2021
https://dl.acm.org/doi/10.1145/3477083.3480155
Hu JIosifescu ALiKamWa RBanerjee SMottola LZhou X(2021)LensCapProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3458864.3467676(14-27)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3458864.3467676
Kodukula VShearer ANguyen VLingutla SLiu YLiKamWa RSherwood TBerger EKozyrakis C(2021)Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low powerProceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3445814.3446737(573-586)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3445814.3446737
Show More Cited By

Index Terms

Characterizing the Reconfiguration Latency of Image Sensor Resolution on Android Devices
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Displays and imagers

Recommendations

Banner -- An Image Sensor Reconfiguration Framework for Seamless Resolution-based Tradeoffs (video)
MobiSys '19: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services

Mobile vision systems would benefit from the ability to situationally sacrifice image resolution to save system energy when imaging detail is unnecessary. Unfortunately, any change in sensor resolution leads to a substantial pause in frame delivery -- ...
An opportunistic reconfiguration strategy for environmentally powered devices
CF '06: Proceedings of the 3rd conference on Computing frontiers

Environmental energy is becoming a feasible alternative to traditional energy sources for ultra low-power devices such as sensor nodes and smart watches. Moreover, the increasing need for flexibility and reconfigurability of such devices makes its ...
A Reconfigurable Hardware Accelerated Platform for Clustered Wireless Sensor Networks
ICPADS '12: Proceedings of the 2012 IEEE 18th International Conference on Parallel and Distributed Systems

With the advent of the FPGA technology, a reconfigurable platform becomes possible to enhance the capability and adaptability of wireless sensor nodes while reducing the energy consumption. In this paper, we investigate the application of reconfigurable ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotMobile '18: Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications

February 2018

130 pages

ISBN:9781450356305

DOI:10.1145/3177102

General Chair:
Minkyong Kim
Samsung Electronics, USA
,
Program Chair:
Aruna Balasubramanian
Stony Brook, NY, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

HotMobile '18

Sponsor:

SIGMOBILE

HotMobile '18: The 19th International Workshop on Mobile Computing Systems and Applications

February 12 - 13, 2018

Arizona, Tempe, USA

Acceptance Rates

HotMobile '18 Paper Acceptance Rate 19 of 65 submissions, 29%;

Overall Acceptance Rate 96 of 345 submissions, 28%

Upcoming Conference

HOTMOBILE '25

Sponsor:
sigmobile

The 26th International Workshop on Mobile Computing Systems and Applications

February 26 - 27, 2025

La Quinta , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
410
Total Downloads

Downloads (Last 12 months)78
Downloads (Last 6 weeks)8

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu JShaikh ABahremand ALiKamWa R(2021)Characterizing real-time dense point cloud capture and streaming on mobile devicesProceedings of the 3rd ACM Workshop on Hot Topics in Video Analytics and Intelligent Edges10.1145/3477083.3480155(1-6)Online publication date: 25-Oct-2021
https://dl.acm.org/doi/10.1145/3477083.3480155
Hu JIosifescu ALiKamWa RBanerjee SMottola LZhou X(2021)LensCapProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3458864.3467676(14-27)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3458864.3467676
Kodukula VShearer ANguyen VLingutla SLiu YLiKamWa RSherwood TBerger EKozyrakis C(2021)Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low powerProceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3445814.3446737(573-586)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3445814.3446737
LiKamWa RHu JKodukula VLiu Y(2021)Adaptive Resolution-Based Tradeoffs for Energy-Efficient Visual Computing SystemsIEEE Pervasive Computing10.1109/MPRV.2021.305252820:2(18-26)Online publication date: 1-Apr-2021
https://doi.org/10.1109/MPRV.2021.3052528
Hu JShearer ARajagopalan SLiKamWa RSong JKim MLane NBalan R(2019)Banner -- An Image Sensor Reconfiguration Framework for Seamless Resolution-based Tradeoffs (video)Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3307334.3328594(705-706)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3307334.3328594
Hu JShearer ARajagopalan SLiKamWa RSong JKim MLane NBalan R(2019)BannerProceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3307334.3326092(236-248)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3307334.3326092
Lubana EDick R(2018)Digital Foveation: An Energy-Aware Machine Vision FrameworkIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2018.285834037:11(2371-2380)Online publication date: Nov-2018
https://doi.org/10.1109/TCAD.2018.2858340

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

EPUB

View this article in ePub.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents