research-article

Securing secure aggregation: mitigating multi-round privacy leakage in federated learning

AUTHORs:

A. Salman AvestimehrAuthors Info & Claims

AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence

Article No.: 1108, Pages 9864 - 9873

https://doi.org/10.1609/aaai.v37i8.26177

Published: 07 February 2023 Publication History

Abstract

Secure aggregation is a critical component in federated learning (FL), which enables the server to learn the aggregate model of the users without observing their local models. Conventionally, secure aggregation algorithms focus only on ensuring the privacy of individual users in a single training round. We contend that such designs can lead to significant privacy leakages over multiple training rounds, due to partial user participation at each round of FL. In fact, we show that the conventional random user selection strategies in FL lead to leaking users' individual models within number of rounds that is linear in the number of users. To address this challenge, we introduce a secure aggregation framework, Multi-RoundSecAgg, with multi-round privacy guarantees. In particular, we introduce a new metric to quantify the privacy guarantees of FL over multiple training rounds, and develop a structured user selection strategy that guarantees the long-term privacy of each user (over any number of training rounds). Our framework also carefully accounts for the fairness and the average number of participating users at each round. Our experiments on MNIST, CIFAR-10 and CIFAR-100 datasets in the IID and the non-IID settings demonstrate the performance improvement over the baselines in terms of privacy protection and test accuracy.

References

[1]

Abadi, M.; Chu, A.; Goodfellow, I.; McMahan, H. B.; Mironov, I.; Talwar, K.; and Zhang, L. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 308-318.

[2]

Bell, J. H.; Bonawitz, K. A.; Gascón, A.; Lepoint, T.; and Raykova, M. 2020. Secure single-server aggregation with (poly) logarithmic overhead. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 1253-1269.

[3]

Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedone, A.; McMahan, H. B.; Patel, S.; Ramage, D.; Segal, A.; and Seth, K. 2017. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 1175-1191.

[4]

Bonawitz, K.; Kairouz, P.; McMahan, B.; and Ramage, D. 2021. Federated Learning and Privacy: Building privacy-preserving systems for machine learning and data science on decentralized data. Queue, 19(5): 87-114.

Digital Library

[5]

Chen, W.; Horvath, S.; and Richtarik, P. 2020. Optimal Client Sampling for Federated Learning. arXiv preprint arXiv:2010.13723.

[6]

Cho, Y. J.; Gupta, S.; Joshi, G.; and Yagan, O. 2020. Bandit-based Communication-Efficient Client Selection Strategies for Federated Learning. arXiv preprint arXiv:2012.08009.

[7]

Cho, Y. J.; Wang, J.; and Joshi, G. 2020. Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies. arXiv preprint arXiv:2010.01243.

[8]

Choi, B.; Sohn, J.-y.; Han, D.-J.; and Moon, J. 2020. Communication-Computation Efficient Secure Aggregation for Federated Learning. arXiv preprint arXiv:2012.05433.

[9]

Deer, A.; Ali, R. E.; and Avestimehr, A. S. 2022. On multi-round Privacy in Federated Learning. In 2022 56th Asilomar conference on signals, systems, and computers. IEEE.

[10]

Dwork, C.; Roth, A.; et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science, 9(3-4): 211-407.

[11]

Elkordy, A. R.; and Avestimehr, A. S. 2020. Secure aggregation with heterogeneous quantization in federated learning. arXiv preprint arXiv:2009.14388.

[12]

Fredrikson, M.; Jha, S.; and Ristenpart, T. 2015. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 1322-1333.

[13]

Geiping, J.; Bauermeister, H.; Droge, H.; and Moeller, M. 2020. Inverting Gradients-How easy is it to break privacy in federated learning? arXiv preprint arXiv:2003.14053.

[14]

Hsu, T.-M. H.; Qi, H.; and Brown, M. 2019. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335.

[15]

Kadhe, S.; Rajaraman, N.; Koyluoglu, O. O.; and Ramchandran, K. 2020. FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning. arXiv preprint arXiv:2009.11248.

[16]

Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis, M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.; Cummings, R.; et al. 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977.

[17]

Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis, M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.; Cummings, R.; et al. 2021. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1-2): 1-210.

[18]

Karimireddy, S. P.; Kale, S.; Mohri, M.; Reddi, S.; Stich, S.; and Suresh, A. T. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, 5132-5143. PMLR.

[19]

Krizhevsky, A.; and Hinton, G. 2009. Learning multiple layers of features from tiny images. Technical report, Citeseer.

[20]

LeCun, Y.; Cortes, C.; and Burges, C. 2010. MNIST handwritten digit database. http://yann.lecun.com/exdb/mnist. Accessed: 2021-12-01.

[21]

Li, X.; Huang, K.; Yang, W.; Wang, S.; and Zhang, Z. 2019. On the Convergence of FedAvg on Non-IID Data. In International Conference on Learning Representations.

[22]

McMahan, H. B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Int. Conf. on Artificial Int. and Stat. (AISTATS), 1273-1282.

[23]

McMahan, H. B.; Ramage, D.; Talwar, K.; and Zhang, L. 2018. Learning differentially private recurrent language models. Int. Conf. on Learning Representations (ICLR).

[24]

Nasr, M.; Shokri, R.; and Houmansadr, A. 2019. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), 739-753. IEEE.

[25]

Pejó, B.; and Biczók, G. 2020. Quality Inference in Federated Learning with Secure Aggregation. arXiv preprint arXiv:2007.06236.

[26]

Ribero, M.; and Vikalo, H. 2020. Communication-efficient federated learning via optimal client sampling. arXiv preprint arXiv:2007.15197.

[27]

Simonyan, K.; and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

[28]

So, J.; Ali, R. E.; Güler, B.; and Avestimehr, A. S. 2021a. Secure aggregation for buffered asynchronous federated learning. arXiv preprint arXiv:2110.02177.

[29]

So, J.; Ali, R. E.; Guler, B.; Jiao, J.; and Avestimehr, S. 2021b. Securing secure aggregation: Mitigating multi-round privacy leakage in federated learning. arXiv preprint arXiv:2106.03328.

[30]

So, J.; Güler, B.; and Avestimehr, A. S. 2021. Turboaggregate: Breaking the quadratic aggregation barrier in secure federated learning. IEEE Journal on Selected Areas in Information Theory, 2(1): 479-489.

[31]

So, J.; He, C.; Yang, C.-S.; Li, S.; Yu, Q.; E Ali, R.; Guler, B.; and Avestimehr, S. 2022. Lightsecagg: a lightweight and versatile design for secure aggregation in federated learning. Proceedings of Machine Learning and Systems, 4: 694-720.

[32]

Tang, M.; Ning, X.; Wang, Y.; Wang, Y.; and Chen, Y. 2021. FedGP: Correlation-Based Active Client Selection for Heterogeneous Federated Learning. arXiv preprint arXiv:2103.13822.

[33]

Wei, K.; Li, J.; Ding, M.; Ma, C.; Yang, H. H.; Farokhi, F.; Jin, S.; Quek, T. Q.; and Poor, H. V. 2020. Federated learning with differential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security, 15: 3454-3469.

Digital Library

[34]

Yang, C.-S.; So, J.; He, C.; Li, S.; Yu, Q.; and Avestimehr, S. 2021. LightSecAgg: Rethinking Secure Aggregation in Federated Learning. arXiv preprint arXiv:2109.14236.

[35]

Yu, H.; Yang, S.; and Zhu, S. 2019. Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, 5693-5700.

[36]

Zhao, Y.; and Sun, H. 2021. Information Theoretic Secure Aggregation with User Dropouts. arXiv preprint arXiv:2101.07750.

[37]

Zhu, L.; and Han, S. 2020. Deep leakage from gradients. In Federated Learning, 17-31. Springer.

Cited By

Kato FXiong LTakagi SCao YYoshikawa M(2024)Uldp-FL: Federated Learning with Across-Silo User-Level Differential PrivacyProceedings of the VLDB Endowment10.14778/3681954.368196617:11(2826-2839)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681966
Zhang YZeng DLuo JFu XChen GXu ZKing I(2024)A Survey of Trustworthy Federated Learning: Issues, Solutions, and ChallengesACM Transactions on Intelligent Systems and Technology10.1145/367818115:6(1-47)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3678181
Zuo RZheng CLi FZhu LZhang Z(2024)Privacy-Enhanced Prototype-Based Federated Cross-Modal Hashing for Cross-Modal RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367450720:9(1-19)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3674507
Show More Cited By

Recommendations

Information Theoretic Secure Aggregation with User Dropouts
2021 IEEE International Symposium on Information Theory (ISIT)
In the robust secure aggregation problem, a server wishes to learn and only learn the sum of the inputs of a number of users while some users may drop out (i.e., may not respond). The identity of the dropped users is not known a priori and the server ...
Md2 is not Secure Without the Checksum Byte
Special issue: selected areas in cryptography I

In 1989, Ron Rivest introduced the MD2 Message Digest Algorithm which takes as input a message of arbitrary length and produces as output a 128-bit message digest, by appending some redundancy to the message and then iteratively applying a 32 bytes to ...
Can GOST Be Made Secure Against Differential Cryptanalysis?

Nothing can be more mistaken. In this article, the authors review 40 years of development of block ciphers in order to resist differential attacks. They study all ten known sets of GOST S-boxes. It appears that the choice of S-boxes has a limited effect ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence

February 2023

16496 pages

ISBN:978-1-57735-880-0

Copyright © 2023 Association for the Advancement of Artificial Intelligence.

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 07 February 2023

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kato FXiong LTakagi SCao YYoshikawa M(2024)Uldp-FL: Federated Learning with Across-Silo User-Level Differential PrivacyProceedings of the VLDB Endowment10.14778/3681954.368196617:11(2826-2839)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681966
Zhang YZeng DLuo JFu XChen GXu ZKing I(2024)A Survey of Trustworthy Federated Learning: Issues, Solutions, and ChallengesACM Transactions on Intelligent Systems and Technology10.1145/367818115:6(1-47)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3678181
Zuo RZheng CLi FZhu LZhang Z(2024)Privacy-Enhanced Prototype-Based Federated Cross-Modal Hashing for Cross-Modal RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367450720:9(1-19)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3674507
Belal YBen Mokhtar SHaddadi HWang JMashhadi A(2024)Survey of Federated Learning Models for Spatial-Temporal Mobility ApplicationsACM Transactions on Spatial Algorithms and Systems10.1145/366608910:3(1-39)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3666089
Zhou YZheng PCao XHuang JLuo BLiao XXu JKirda ELie D(2024)Two-Tier Data Packing in RLWE-based Homomorphic Encryption for Secure Federated LearningProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690191(2844-2858)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3690191
Jin CYin Cvan Dijk MDuan SMassacci FReiter MZhang HLuo BLiao XXu JKirda ELie D(2024)PG: Byzantine Fault-Tolerant and Privacy-Preserving Sensor Fusion with Guaranteed Output DeliveryProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670343(3272-3286)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670343

View Options

View options

Figures

Tables

Media

View Table of Conten