skip to main content
10.1145/3570361.3613303acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
research-article
Public Access

Squint: A Framework for Dynamic Voltage Scaling of Image Sensors Towards Low Power IoT Vision

Published: 02 October 2023 Publication History

Abstract

Energy-efficient visual sensing is of paramount importance to enable battery-backed low power IoT and mobile applications. Unfortunately, modern image sensors still consume hundreds of milliwatts of power, mainly due to analog readout. This is because current systems always supply a fixed voltage to the sensor's analog circuitry, leading to higher power profiles. In this work, we propose to aggressively scale the analog voltage supplied to the camera as a means to significantly reduce sensor power consumption. To that end, we characterize the power and fidelity implications of analog voltage scaling on three off-the-shelf image sensors. Our characterization reveals that analog voltage scaling reduces sensor power but also degrades image quality. Furthermore, the degradation in image quality situationally affects the task accuracy of vision applications.
We develop a visual streaming pipeline that flexibly allows application developers to dynamically adapt sensor voltage on a frame-by-frame basis. We develop a voltage controller that programmatically generates desired sensor voltage based on application request. We integrate our voltage controller into the existing RPi-based video streaming IoT pipeline. On top of this, we develop runtime support for flexible voltage specification from vision applications. Evaluating the system over a wide range of voltage scaling policies on popular vision tasks reveals that Squint imaging can deliver up to 73% sensor power savings, while maintaining reasonable task fidelity. Our artifacts are available at: https://gitlab.com/squint1/squint-ae-public

References

[1]
Deepak Vasisht, Zerina Kapetanovic, Jongho Won, Xinxin Jin, Ranveer Chandra, Sudipta Sinha, Ashish Kapoor, Madhusudhan Sudarshan, and Sean Stratman. {FarmBeats}: An {IoT} platform for {Data-Driven} agriculture. In 14th USENIX Symp. on Networked Systems Design and Implementation (NSDI 17), 2021.
[2]
Mauricio Delbracio, Damien Kelly, Michael S. Brown, and Peyman Milanfar. Mobile computational photography: A tour. Annual Review of Vision Science, 2021.
[3]
Cindy S Bick, Inhee Lee, Trevor Coote, Amanda E Haponski, David Blaauw, and Diarmaid Ó Foighil. Millimeter-sized smart sensors reveal that a solar refuge protects tree snail partula hyalina from extirpation. Nature's Communications Biology, 2021.
[4]
Vikram Iyer, Ali Najafi, Johannes James, Sawyer Fuller, and Shyamnath Gollakota. Wireless steerable vision for live insects and insect-scale robots. Science robotics, 2020.
[5]
Tom's guide. Magic Leap 2 release date, price and all the new features. https://www.tomsguide.com/news/magic-leap-2-everything-weve-heard-so-far.
[6]
Mark Weiser, Brent Welch, Alan Demers, and Scott Shenker. Scheduling for reduced cpu energy. In Mobile Computing, pages 449--471. Springer, 1994.
[7]
Image Sensors World. Era of 2.2V Supply Coming? https://image-sensors-world.blogspot.com/2021/09/era-of-22v-supply-coming.html.
[8]
Image Sensors World. Samsung Announces 200MP 0.64um Pixel Mobile Sensor. https://image-sensors-world.blogspot.com/2021/09/samsung-announces-200mp-064um-pixel.html.
[9]
Image Sensors World. Samsung Announces 50MP 1um Pixel Sensor with All-Direction PDAF. https://image-sensors-world.blogspot.com/2021/09/samsung-announces-50mp-1um-pixel-sensor.html.
[10]
Jaehyuk Choi, Jungsoon Shin, Dongwu Kang, and Du-Sik Park. Always-on cmos image sensor for mobile and wearable devices. IEEE Journal of Solid-State Circuits (JSSC), 2015.
[11]
Robert LiKamWa, Bodhi Priyantha, Matthai Philipose, Lin Zhong, and Paramvir Bahl. Energy characterization and optimization of image sensing toward continuous mobile vision. In Proc. of the 11th annual international conf on Mobile systems, applications, and services, 2013.
[12]
Kazuya Kitamura, Toshihisa Watabe, Takehide Sawamoto, Tomohiko Kosugi, Tomoyuki Akahori, Tetsuya Iida, Keigo Isobe, Takashi Watanabe, Hiroshi Shimamoto, Hiroshi Ohtake, et al. A 33-megapixel 120-frames-per-second 2.5-watt cmos image sensor with column-parallel two-stage cyclic analog-to-digital converters. IEEE Transactions on Electron Devices, 2012.
[13]
Jaehyuk Choi, Seokjun Park, Jihyun Cho, and Euisik Yoon. An energy/illumination-adaptive cmos image sensor with reconfigurable modes of operations. IEEE Journal of Solid-State Circuits, 2015.
[14]
Isao Takayanagi, Miho Shirakawa, Koji Mitani, Masayuki Sugawara, Steinar Iversen, Jørgen Moholt, Junichi Nakamura, and Eric R Fossum. A 1.25-inch 60-frames/s 8.3-m-pixel digital-output cmos image sensor. IEEE Journal of Solid-State Circuits, 2005.
[15]
Che-I Lin, Cheng-Hsiao Lai, and Ya-Chin King. A four transistor cmos active pixel sensor with high dynamic range operation. In Proceedings of Asia-Pacific Conference on Advanced System Integrated Circuits. IEEE, 2004.
[16]
Eric R Fossum and Donald B Hondongwa. A review of the pinned photodiode for ccd and cmos image sensors. IEEE Journal of the electron devices society, 2014.
[17]
Sony. IMX219PQH5-C. https://www.arducam.com/downloads/modules/RaspberryPi_camera/IMX219DS.PDF.
[18]
OmniVision. OV5647 datasheet. https://www.uctronics.com/download/Image_Sensor/OV5647_DS.pdf.
[19]
OmniVision. PYTHON 1.3/0.5/0.3 MegaPixels Global Shutter CMOS Image Sensors. https://www.onsemi.com/pdf/datasheet/noip1sn1300a-d.pdf.
[20]
R Jacob Baker. CMOS: circuit design, layout, and simulation. John Wiley & Sons, 2019.
[21]
John L Hennessey and David A Patterson. Computer Architecture: A Quantitative Approach. Elsevier, 2018.
[22]
Albert Theuwissen. How To Measure Temporal Noise in Dark ? https://harvestimaging.com/blog/?p=994.
[23]
Albert Theuwissen. How to measure the average dark signal ? https://harvestimaging.com/blog/?p=795.
[24]
Marc Ebner. Combining white-patch retinex and the gray world assumption to achieve color constancy for multiple illuminants. In Joint Pattern Recognition Symposium, pages 60--67. Springer, 2003.
[25]
M Ebner. The gray world assumption. Color Constancy; John Wiley & Sons: Chichester, UK, pages 106--108, 2007.
[26]
Dongliang Cheng, Dilip K Prasad, and Michael S Brown. Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. JOSA A, 2014.
[27]
Joost Van De Weijer, Theo Gevers, and Arjan Gijsenij. Edge-based color constancy. IEEE Transactions on image processing, 2007.
[28]
Vadim Ziyadinov and Maxim Tereshonok. Noise immunity and robustness study of image recognition using a convolutional neural network. Sensors, 2022.
[29]
Sony. IMX219 datasheet. https://github.com/rellimmot/Sony-IMX219-Raspberry-Pi-V2-CMOS/blob/master/RASPBERRY%20PI%20CAMERA%20V2%20DATASHEET%20IMX219PQH5_7.0.0_Datasheet_XXX.PDF.
[30]
Micheal Promonet. v4l2rtspserver. https://github.com/mpromonet/v4l2rtspserver.
[31]
Microchip. MCP4725, 12-Bit Digital-to-Analog Converter with EEP-ROM Memory in SOT-23-6. https://ww1.microchip.com/downloads/aemDocuments/documents/OTH/ProductDocuments/DataSheets/22039d.pdf.
[32]
Texas Instruments. OPA55x High-Voltage, High-Current Operational Amplifiers. https://www.ti.com/lit/gpn/OPA551.
[33]
Wikipedia. Representational state transfer. https://en.wikipedia.org/wiki/Representational_state_transfer.
[34]
Roy Thomas Fielding. Architectural styles and the design of network-based software architectures. University of California, Irvine, 2000.
[35]
Wikipedia. Amazon Go. https://en.wikipedia.org/wiki/Amazon_Go.
[36]
Wikipedia. Microsoft HoloLens. https://en.wikipedia.org/wiki/Microsoft_HoloLens.
[37]
Frank Dehmelt. Adaptive (dynamic) voltage (frequency) scaling---motivation and implementation. Texas Instruments - Application Report, pages 1--10, 2014.
[38]
Mohamed Elgebaly and Manoj Sachdev. Variation-aware adaptive voltage scaling system. IEEE Tran. on Very Large Scale Integration (VLSI) Systems, 2007.
[39]
THE HOME HACKS DIY. How Much Power (Watts) Does a VR Headset Use? https://www.thehomehacksdiy.com/how-much-power-watts-does-a-vr-headset-use.
[40]
TrailGuard-AI-in-the-News. TrailGuard AI. https://www.resolve.ngo/blog/TrailGuard-AI-in-the-News.htm.
[41]
Anantha P Chandrakasan, Samuel Sheng, and Robert W Brodersen. Low-power cmos digital design. IEICE Transactions on Electronics, 1992.
[42]
Mark A Horowitz. Self-clocked structures for low power systems. ARPA semi-annual report, 1993.
[43]
Masakatsu Nakai, Satoshi Akui, Katsunori Seno, Tetsumasa Meguro, Takahiro Seki, Tetsuo Kondo, Akihiko Hashiguchi, Hirokazu Kawahara, Kazuo Kumano, and Masayuki Shimura. Dynamic voltage and frequency management for a low-power embedded microprocessor. IEEE journal of solid-state Circuits, 2005.
[44]
Jan M Rabaey, AP Chandrakasan, and B Nikolic. Digital integrated circuits, vol. 996, 1996.
[45]
Wonyoung Kim, Meeta S Gupta, Gu-Yeon Wei, and David Brooks. System level analysis of fast, per-core dvfs using on-chip switching regulators. In IEEE 14th Int Symp. on High Performance Computer Architecture, 2008.
[46]
Masato Osawa, Shuzo Hiraide, Shunsuke Suzuki, Hideki Kato, Kosei Tamiya, Yasunari Harada, Kuba Raczkowski, Jean-Luc Bacq, Peter Van Wesemael, Mingxu Liu, et al. An adaptive frame image sensor with fine-grained power management for ultra-low power internet of things application. In IEEE 44th European Solid State Circuits Conference (ESSCIRC), 2018.
[47]
Hirotaka Murakami, Eric Bohannon, John Childs, Grace Gui, Eric Moule, Katsuhiko Hanzawa, Tomofumi Koda, Chiaki Takano, Toshimasa Shimizu, Yuki Takizawa, et al. A 4.9 mpixel programmable-resolution multi-purpose cmos image sensor for computer vision. In IEEE Int. Solid-State Circuits Conference (ISSCC), 2022.
[48]
Saman Naderiparizi, Aaron N Parks, Zerina Kapetanovic, Benjamin Ransford, and Joshua R Smith. Wispcam: A battery-free rfid camera. In IEEE Intl. Conf on RFID (RFID), 2015.
[49]
Saman Naderiparizi, Mehrdad Hessar, Vamsi Talla, Shyamnath Gollakota, and Joshua R Smith. Towards {Battery-Free} {HD} video streaming. In 15th USENIX Symp. on Networked Systems Design and Implementation (NSDI), 2018.
[50]
Colleen Josephson, Lei Yang, Pengyu Zhang, and Sachin Katti. Wireless computer vision using commodity radios. In Proc. of the 18th Intl. Conf. on Information Processing in Sensor Networks (IPSN), 2019.
[51]
Saman Naderiparizi, Pengyu Zhang, Matthai Philipose, Bodhi Priyantha, Jie Liu, and Deepak Ganesan. Glimpse: A programmable early-discard camera architecture for continuous mobile vision. In Proc. of the 15th Annual Intl Conf. on Mobile Systems, Applications, and Services (MobiSys), 2017.
[52]
Matteo Nardello, Harsh Desai, Davide Brunelli, and Brandon Lucia. Camaroptera: A batteryless long-range remote visual sensing system. In Proc. of the 7th Intl. Workshop on Energy Harvesting & Energy-Neutral Sensing Systems, 2019.
[53]
Venkatesh Kodukula, Saad Katrawala, Britton Jones, Carole-Jean Wu, and Robert LiKamWa. Dynamic temperature management of near-sensor processing for energy-efficient high-fidelity imaging. Sensors, 2021.
[54]
Jinhan Hu, Alexander Shearer, Saranya Rajagopalan, and Robert LiKamWa. Banner: An image sensor reconfiguration framework for seamless resolution-based tradeoffs. In Proc. of the 17th Annual Int Conf on Mobile Systems, Applications, and Services, 2019.
[55]
Venkatesh Kodukula, Alexander Shearer, Van Nguyen, Srinivas Lingutla, Yifei Liu, and Robert LiKamWa. Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low power. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, 2021.

Cited By

View all

Index Terms

  1. Squint: A Framework for Dynamic Voltage Scaling of Image Sensors Towards Low Power IoT Vision

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ACM MobiCom '23: Proceedings of the 29th Annual International Conference on Mobile Computing and Networking
    October 2023
    1605 pages
    ISBN:9781450399906
    DOI:10.1145/3570361
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. image sensors
    2. voltage scaling
    3. IoT
    4. AR/VR
    5. energy efficiency

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ACM MobiCom '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 440 of 2,972 submissions, 15%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)271
    • Downloads (Last 6 weeks)35
    Reflects downloads up to 23 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media