research-article

Secure Neural Network Inference as a Service with Resource-Constrained Clients

Authors:

Zoltán Ádám MannAuthors Info & Claims

UCC '23: Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud Computing

Article No.: 8, Pages 1 - 10

https://doi.org/10.1145/3603166.3632132

Published: 04 April 2024 Publication History

Abstract

Applying services computing to neural networks, a service provider may provide inference with a pre-trained neural network as a service. Clients use the service to get the neural network's output on their input. To protect sensitive data, secure neural network inference (SNNI) entails that only the client learns the output; the input remains the client's secret and the neural network's parameters remain the service provider's secret. Several SNNI approaches were proposed and evaluated in environments where both service providers and clients used powerful computers.

In many real settings, for instance in edge computing, client devices are resource-constrained. This paper is the first to investigate the impact of client-side resource constraints on SNNI. We perform experiments with two state-of-the-art SNNI approaches and three neural networks. We vary the compute and memory capacity of the client device and measure the impact on inference time. Our findings show that client-side resource constraints significantly impact the performance and even the applicability of SNNI approaches. The results indicate the limits of current SNNI approaches for resource-constrained clients. Based on the results, we identify research directions to improve SNNI for resource-constrained clients.

References

[1]

Laith Alzubaidi, Jinglan Zhang, Amjad J. Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al-Shamma, J. Santamaría, Mohammed A. Fadhel, Muthana Al-Amidie, and Laith Farhan. 2021. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data 8, 1 (2021).

[2]

Dhouha Ayed, Paul-Andrei Dragan, Edith Félix, Zoltán Adám Mann, Eliot Salant, Robert Seidl, Anestis Sidiropoulos, Steve Taylor, and Ricardo Vitorino. 2022. Protecting sensitive data in the cloud-to-edge continuum: The FogProtect approach. In 22nd International Symposium on Cluster, Cloud and Internet Computing (CCGrid). IEEE, 279--288.

[3]

Ruhi Kiran Bajaj, Rebecca Mary Meiring, and Fernando Beltran. 2023. Co-Design, Development, and Evaluation of a Health Monitoring Tool Using Smartwatch Data: A Proof-of-Concept Study. Future Internet 15, 3 (2023), 111.

[4]

Mauro Barni, Claudio Orlandi, and Alessandro Piva. 2006. A privacy-preserving protocol for neural-network-based computation. In Proceedings of the 8th Workshop on Multimedia and Security. 146--151.

Digital Library

[5]

Chris M. Bishop. 1994. Neural networks and their applications. Review of Scientific Instruments 65, 6 (1994), 1803--1832.

[6]

Daphnee Chabal, Dolly Sapra, and Zoltán Ádám Mann. 2023. On Achieving Privacy-Preserving State-of-the-Art Edge Intelligence. In 4th AAAI Workshop on Privacy-Preserving Artificial Intelligence (PPAI-23).

[7]

Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In 33rd International Conference on Machine Learning. 201--210.

[8]

Gareth Halfacree. 2020. Raspberry Pi 4 B: How Much RAM Do You Really Need? https://www.tomshardware.com/news/raspberry-pi-4-how-much-ram

[9]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770--778.

[10]

Jiahui Hou, Huiqi Liu, Yunxin Liu, Yu Wang, Peng-Jun Wan, and Xiang-Yang Li. 2021. Model Protection: Real-time privacy-preserving inference service for model privacy at the edge. IEEE Transactions on Dependable and Secure Computing 19, 6 (2021), 4270--4284.

[11]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700--4708.

[12]

Kai Huang, Ximeng Liu, Shaojing Fu, Deke Guo, and Ming Xu. 2019. A lightweight privacy-preserving CNN feature extraction framework for mobile sensing. IEEE Transactions on Dependable and Secure Computing 18, 3 (2019), 1441--1455.

[13]

Zhicong Huang, Wen-jie Lu, Cheng Hong, and Jiansheng Ding. 2022. Cheetah: Lean and fast secure two-party deep neural network inference. In 31st USENIX Security Symposium. 809--826.

[14]

Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv preprint arXiv:1602.07360.

[15]

Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18). 1651--1669.

[16]

Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. 2020. CrypTFlow: Secure TensorFlow inference. In IEEE Symposium on Security and Privacy (SP). IEEE, 336--353.

[17]

Clemens Lachner, Zoltán Adám Mann, and Schahram Dustdar. 2021. Towards understanding the adaptation space of AI-assisted data protection for video analytics at the edge. In 41st International Conference on Distributed Computing Systems Workshops (ICDCSW). IEEE, 7--12.

[18]

Byron C. Lewis and Albert E. Crews. 1985. The Evolution of Benchmarking as a Computer Performance Evaluation Technique. MIS Quarterly 9, 1 (1985), 7.

Digital Library

[19]

Jiarui Li, Zhuosheng Zhang, Shucheng Yu, and Jiawei Yuan. 2022. Improved Secure Deep Neural Network Inference Offloading with Privacy-Preserving Scalar Product Evaluation for Edge Computing. Applied Sciences 12, 18 (2022), 9010.

[20]

Jian Liu, Mika Juuti, Yao Lu, and Nadarajah Asokan. 2017. Oblivious neural network predictions via MiniONN transformations. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 619--631.

Digital Library

[21]

Xiaoning Liu, Yifeng Zheng, Xingliang Yuan, and Xun Yi. 2023. Securely Outsourcing Neural Network Inference to the Cloud With Lightweight Techniques. IEEE Transactions on Dependable and Secure Computing 20, 1 (2023), 620--636.

[22]

Zoltán Ádám Mann, Andreas Metzger, Johannes Prade, and Robert Seidl. 2019. Optimized application deployment in the fog. In 17th International Conference on Service-Oriented Computing (ICSOC). Springer, 283--298.

Digital Library

[23]

Zoltán Ádám Mann, Christian Weinert, Daphnee Chabal, and Joppe W. Bos. 2023. Towards Practical Secure Neural Network Inference: The Journey So Far and the Road Ahead. Comput. Surveys (2023), accepted.

[24]

Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. 2020. Delphi: A Cryptographic Inference Service for Neural Networks. In USENIX Security Symposium. 2505--2522.

[25]

Payman Mohassel and Yupeng Zhang. 2017. SecureML: A system for scalable privacy-preserving machine learning. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 19--38.

[26]

Claudio Orlandi, Alessandro Piva, and Mauro Barni. 2007. Oblivious neural network computing via homomorphic encryption. EURASIP Journal on Information Security 2007 (2007), 1--11.

Digital Library

[27]

Seyed Ali Osia, Ali Shahin Shamsabadi, Sina Sajadmanesh, Ali Taheri, Kleomenis Katevas, Hamid R Rabiee, Nicholas D Lane, and Hamed Haddadi. 2020. A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet of Things Journal 7, 5 (2020), 4505--4518.

[28]

Deevashwer Rathee, Mayank Rathee, Rahul Kranti Kiran Goli, Divya Gupta, Rahul Sharma, Nishanth Chandran, and Aseem Rastogi. 2021. SiRnn: A math library for secure RNN inference. In IEEE Symposium on Security and Privacy (SP). IEEE, 1003--1020.

[29]

Deevashwer Rathee, Mayank Rathee, Nishant Kumar, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. 2020. CrypTFlow2: Practical 2-party secure inference. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (CCS). 325--342.

Digital Library

[30]

M Sadegh Riazi, Mohammad Samragh, Hao Chen, Kim Laine, Kristin Lauter, and Farinaz Koushanfar. 2019. XONN: XNOR-based oblivious deep neural network inference. In 28th USENIX Security Symposium. 1501--1518.

[31]

M Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. 2018. Chameleon: A hybrid secure computation framework for machine learning applications. In Proceedings of the 2018 Asia Conference on Computer and Communications Security. 707--721.

Digital Library

[32]

Mauro Ribeiro, Katarina Grolinger, and Miriam A.M. Capretz. 2015. MLaaS: Machine Learning as a Service. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA). IEEE.

[33]

Bita Darvish Rouhani, M Sadegh Riazi, and Farinaz Koushanfar. 2018. DeepSecure: Scalable provably-secure deep learning. In 55th Annual Design Automation Conference.

Digital Library

[34]

Tjerk Timan and Zoltan Mann. 2021. Data protection in the era of artificial intelligence: trends, existing solutions and recommendations for privacy-preserving technologies. In The Elements of Big Data Value: Foundations of the Research and Innovation Ecosystem. Springer, 153--175.

[35]

Mengyao Zheng, Dixing Xu, Linshan Jiang, Chaojie Gu, Rui Tan, and Peng Cheng. 2019. Challenges of privacy-preserving machine learning in IoT. In 1st Intl. Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things. 1--7.

Digital Library

Cited By

Islam TOprescu AMann ZKlous S(2024)A Framework to Optimize the Energy Cost of Securing Neural Network Inference2024 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Computing & Communications (GreenCom) and IEEE Cyber, Physical & Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics62450.2024.00073(339-346)Online publication date: 19-Aug-2024
https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics62450.2024.00073
Batool KAnwar SMann Z(2024)SecFePAS: Secure Facial-Expression-Based Pain Assessment with Deep Learning at the Edge2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00046(417-424)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00046
Zhang EMann Z(2024)Predicting the Execution Time of Secure Neural Network InferenceICT Systems Security and Privacy Protection10.1007/978-3-031-65175-5_34(481-494)Online publication date: 26-Jul-2024
https://doi.org/10.1007/978-3-031-65175-5_34

Index Terms

Secure Neural Network Inference as a Service with Resource-Constrained Clients

Recommendations

Towards Practical Secure Neural Network Inference: The Journey So Far and the Road Ahead
Neural networks (NNs) have become one of the most important tools for artificial intelligence. Well-designed and trained NNs can perform inference (e.g., make decisions or predictions) on unseen inputs with high accuracy. Using NNs often involves ...
CHET: an optimizing compiler for fully-homomorphic neural-network inferencing
PLDI 2019: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

Fully Homomorphic Encryption (FHE) refers to a set of encryption schemes that allow computations on encrypted data without requiring a secret key. Recent cryptographic advances have pushed FHE into the realm of practical applications. However, ...
GMDH-type neural network algorithm with a feedback loop for structural identification of RBF neural network

In this paper, a Group Method of Data Handling (GMDH)-type neural network algorithm with a feedback loop for structural identification of Radial Basis Function (RBF) neural network is proposed. In case of the GMDH-type neural network, the network ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UCC '23: Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud Computing

December 2023

502 pages

ISBN:9798400702341

DOI:10.1145/3603166

Co-chairs:
Omer Rana,
Massimo Villari,
Program Co-chairs:
Song Fu,
Lorenzo Carnevale

Copyright © 2023 Copyright is held by the owner/author(s). Publication rights licensed to ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2024

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UCC '23

Sponsor:

SIGARCH

UCC '23: IEEE/ACM 16th International Conference on Utility and Cloud Computing

December 4 - 7, 2023

Taormina (Messina), Italy

Acceptance Rates

Overall Acceptance Rate 38 of 125 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
75
Total Downloads

Downloads (Last 12 months)75
Downloads (Last 6 weeks)7

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Islam TOprescu AMann ZKlous S(2024)A Framework to Optimize the Energy Cost of Securing Neural Network Inference2024 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Computing & Communications (GreenCom) and IEEE Cyber, Physical & Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics62450.2024.00073(339-346)Online publication date: 19-Aug-2024
https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics62450.2024.00073
Batool KAnwar SMann Z(2024)SecFePAS: Secure Facial-Expression-Based Pain Assessment with Deep Learning at the Edge2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00046(417-424)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00046
Zhang EMann Z(2024)Predicting the Execution Time of Secure Neural Network InferenceICT Systems Security and Privacy Protection10.1007/978-3-031-65175-5_34(481-494)Online publication date: 26-Jul-2024
https://doi.org/10.1007/978-3-031-65175-5_34

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten