research-article

Deep Learning with Differential Privacy

Authors:

Ian Goodfellow,

H. Brendan McMahan,

Li ZhangAuthors Info & Claims

CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security

Pages 308 - 318

https://doi.org/10.1145/2976749.2978318

Published: 24 October 2016 Publication History

Abstract

Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains. Often, the training of models requires large, representative datasets, which may be crowdsourced and contain sensitive information. The models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.

References

[1]

CIFAR-10 and CIFAR-100 datasets. www.cs.toronto.edu/~kriz/cifar.html.

[2]

TensorFlow convolutional neural networks tutorial. www.tensor ow.org/tutorials/deepcnn.

[3]

TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.

[4]

M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. CoRR, abs/1607.00133, 2016.

Digital Library

[5]

C. C. Aggarwal. On k-anonymity and the curse of dimensionality. In VLDB, pages 901--909, 2005.

Digital Library

[6]

R. Agrawal and R. Srikant. Privacy-preserving data mining. In SIGMOD, pages 439--450. ACM, 2000.

Digital Library

[7]

R. Bassily, K. Nissim, A. Smith, T. Steinke, U. Stemmer, and J. Ullman. Algorithmic stability for adaptive data analysis. In STOC, pages 1046--1059. ACM, 2016.

Digital Library

[8]

R. Bassily, A. D. Smith, and A. Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In FOCS, pages 464--473. IEEE, 2014.

Digital Library

[9]

A. Beimel, H. Brenner, S. P. Kasiviswanathan, and K. Nissim. Bounds on the sample complexity for private learning and private data release. Machine Learning, 94(3):401--437, 2014.

Digital Library

[10]

J. Brickell and V. Shmatikov. The cost of privacy: Destruction of data-mining utility in anonymized data publishing. In KDD, pages 70--78. ACM, 2008.

Digital Library

[11]

M. Bun and T. Steinke. Concentrated di erential privacy: Simpli cations, extensions, and lower bounds. CoRR, abs/1605.02065, 2016.

[12]

K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. J. Machine Learning Research, 12:1069--1109, 2011.

Digital Library

[13]

R. Collobert, K. Kavukcuoglu, and C. Farabet. Torch7: A Matlab-like environment for machine learning. In BigLearn, NIPS Workshop, number EPFL-CONF-192376, 2011.

[14]

D. D. Cox and N. Pinto. Beyond simple features: A large-scale feature search approach to unconstrained face recognition. In FG 2011, pages 8--15. IEEE, 2011.

[15]

D. Silver, A. Huang, C. J. Maddison et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484--489, 2016.

[16]

A. Daniely, R. Frostig, and Y. Singer. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity. CoRR, abs/1602.05897, 2016.

[17]

J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Machine Learning Research, 12:2121--2159, July 2011.

Digital Library

[18]

C. Dwork. A rm foundation for private data analysis. Commun. ACM, 54(1):86--95, Jan. 2011.

Digital Library

[19]

C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486--503. Springer, 2006.

Digital Library

[20]

C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371--380. ACM, 2009.

Digital Library

[21]

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pages 265--284. Springer, 2006.

Digital Library

[22]

C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3--4):211--407, 2014.

Digital Library

[23]

C. Dwork and G. N. Rothblum. Concentrated differential privacy. CoRR, abs/1603.01887, 2016.

[24]

C. Dwork, G. N. Rothblum, and S. Vadhan. Boosting and differential privacy. In FOCS, pages 51--60. IEEE, 2010.

Digital Library

[25]

C. Dwork, K. Talwar, A. Thakurta, and L. Zhang. Analyze Gauss: Optimal bounds for privacy-preserving principal component analysis. In STOC, pages 11--20. ACM, 2014.

Digital Library

[26]

M. Fredrikson, S. Jha, and T. Ristenpart. Model inversion attacks that exploit con dence information and basic countermeasures. In CCS, pages 1322--1333. ACM, 2015.

Digital Library

[27]

I. Goodfellow. Efficient per-example gradient computations. CoRR, abs/1510.01799v2, 2015.

[28]

B. Graham. Fractional max-pooling. CoRR, abs/1412.6071, 2014.

[29]

A. Gupta, K. Ligett, F. McSherry, A. Roth, and K. Talwar. Di erentially private combinatorial optimization. In SODA, pages 1106--1125, 2010.

Digital Library

[30]

K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In ICCV, pages 1026--1034. IEEE, 2015.

Digital Library

[31]

R. Ierusalimschy, L. H. de Figueiredo, and W. Filho. Lua|an extensible extension language. Software: Practice and Experience, 26(6):635--652, 1996.

Digital Library

[32]

K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In ICCV, pages 2146--2153. IEEE, 2009.

[33]

R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In NIPS, pages 315--323, 2013.

Digital Library

[34]

P. Kairouz, S. Oh, and P. Viswanath. The composition theorem for di erential privacy. In ICML, pages 1376--1385. ACM, 2015.

[35]

S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. D. Smith. What can we learn privately? SIAM J. Comput., 40(3):793--826, 2011.

Digital Library

[36]

D. Kifer, A. D. Smith, and A. Thakurta. Private convex optimization for empirical risk minimization with applications to high-dimensional regression. In COLT, pages 25.1--25.40, 2012.

[37]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1097--1105, 2012.

Digital Library

[38]

Y. LeCun, L. Bottou, Y. Bengio, and P. Ha ner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 1998.

[39]

Y. Lindell and B. Pinkas. Privacy preserving data mining. In CRYPTO, pages 36--54. Springer, 2000.

Digital Library

[40]

C. J. Maddison, A. Huang, I. Sutskever, and D. Silver. Move evaluation in Go using deep convolutional neural networks. In ICLR, 2015.

[41]

F. McSherry and I. Mironov. Di erentially private recommender systems: Building privacy into the Netflix Prize contenders. In KDD, pages 627--636. ACM, 2009.

Digital Library

[42]

F. D. McSherry. Privacy integrated queries: An extensible platform for privacy-preserving data analysis. In SIGMOD, pages 19--30. ACM, 2009.

Digital Library

[43]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013.

[44]

I. Mironov. Renyi differential privacy. Private communication, 2016.

[45]

Y. Nesterov. Introductory Lectures on Convex Optimization. A Basic Course. Springer, 2004.

[46]

J. Pennington, R. Socher, and C. D. Manning. GloVe: Global vectors for word representation. In EMNLP, pages 1532--1543, 2014.

[47]

N. Phan, Y. Wang, X. Wu, and D. Dou. Di erential privacy preservation for deep auto-encoders: an application of human behavior prediction. In AAAI, pages 1309--1316, 2016.

[48]

N. Pinto, Z. Stone, T. E. Zickler, and D. Cox. Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on Facebook. In CVPR, pages 35--42. IEEE, 2011.

[49]

R. M. Rogers, A. Roth, J. Ullman, and S. P. Vadhan. Privacy odometers and lters: Pay-as-you-go composition. CoRR, abs/1605.08294, 2016.

[50]

D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 323:533--536, Oct. 1986.

[51]

A. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Ng. On random weights and unsupervised feature learning. In ICML, pages 1089--1096. ACM, 2011.

Digital Library

[52]

R. Shokri and V. Shmatikov. Privacy-preserving deep learning. In CCS, pages 1310--1321. ACM, 2015.

Digital Library

[53]

S. Song, K. Chaudhuri, and A. Sarwate. Stochastic gradient descent with di erentially private updates. In GlobalSIP Conference, 2013.

[54]

L. Sweeney. k-anonymity: A model for protecting privacy. International J. of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(05):557--570, 2002.

Digital Library

[55]

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, pages 1--9. IEEE, 2015.

[56]

S. Tu, R. Roelofs, S. Venkataraman, and B. Recht. Large scale kernel learning using block coordinate descent. CoRR, abs/1602.05310, 2016.

[57]

O. Vinyals, L. Kaiser, T. Koo, S. Petrov, I. Sutskever, and G. E. Hinton. Grammar as a foreign language. In NIPS, pages 2773--2781, 2015.

Digital Library

[58]

X. Wu, A. Kumar, K. Chaudhuri, S. Jha, and J. F. Naughton. Di erentially private stochastic gradient descent for in-RDBMS analytics. CoRR, abs/1606.04722, 2016.

Cited By

Ngoupayou Limbepe ZGai KYu J(2025)Blockchain-Based Privacy-Enhancing Federated Learning in Smart Healthcare: A SurveyBlockchains10.3390/blockchains30100013:1(1)Online publication date: 1-Jan-2025
https://doi.org/10.3390/blockchains3010001
Pan THou JJin XLi CCai XZhou X(2025)Differentially Private Clustered Federated Load Prediction Based on the Louvain AlgorithmAlgorithms10.3390/a1801003218:1(32)Online publication date: 8-Jan-2025
https://doi.org/10.3390/a18010032
Mehmood RLazar MLiang XCorchado JSee S(2025)Editorial: Protecting privacy in neuroimaging analysis: balancing data sharing and privacy preservationFrontiers in Neuroinformatics10.3389/fninf.2024.154312118Online publication date: 7-Jan-2025
https://doi.org/10.3389/fninf.2024.1543121
Show More Cited By

Index Terms

Deep Learning with Differential Privacy
1. Computing methodologies
  1. Machine learning
2. Security and privacy

Recommendations

Differential privacy in deep learning: A literature survey
Abstract
The widespread adoption of deep learning is facilitated in part by the availability of large-scale data for training desirable models. However, these data may involve sensitive personal information, which raises privacy concerns for data ...
Differential privacy in deep learning: Privacy and beyond
Abstract
Motivated by the security risks of deep neural networks, such as various membership and attribute inference attacks, differential privacy has emerged as a promising approach for protecting the privacy of neural networks. As a result, it is ...
Highlights
- Mechanisms: we categorize the existing DP mechanisms' improvements into three categories.
- Upper and lower bounds: we focus on both the upper and lower bounds of DP.
- Beyond privacy: apart from privacy, we summarize other effects of ...
Learning with differential privacy: stability, learnability and the sufficiency and necessity of ERM principle

While machine learning has proven to be a powerful data-driven solution to many real-life problems, its use in sensitive domains has been limited due to privacy concerns. A popular approach known as differential privacy offers provable privacy guarantees,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security

October 2016

1924 pages

ISBN:9781450341394

DOI:10.1145/2976749

General Chairs:
Edgar Weippl
SBA Research, Austria
,
Stefan Katzenbeisser
TU Darmstadt, CYSEC, Germany
,
Program Chairs:
Christopher Kruegel
University of California, Santa Barbara, USA
,
Andrew Myers
Cornell University, USA
,
Shai Halevi
IBM Research, USA

Copyright © 2016 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives International 4.0 License.

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS'16

Sponsor:

SIGSAC

CCS'16: 2016 ACM SIGSAC Conference on Computer and Communications Security

October 24 - 28, 2016

Vienna, Austria

Acceptance Rates

CCS '16 Paper Acceptance Rate 137 of 831 submissions, 16%;

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3,265
Total Citations
View Citations
25,840
Total Downloads

Downloads (Last 12 months)3,516
Downloads (Last 6 weeks)454

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ngoupayou Limbepe ZGai KYu J(2025)Blockchain-Based Privacy-Enhancing Federated Learning in Smart Healthcare: A SurveyBlockchains10.3390/blockchains30100013:1(1)Online publication date: 1-Jan-2025
https://doi.org/10.3390/blockchains3010001
Pan THou JJin XLi CCai XZhou X(2025)Differentially Private Clustered Federated Load Prediction Based on the Louvain AlgorithmAlgorithms10.3390/a1801003218:1(32)Online publication date: 8-Jan-2025
https://doi.org/10.3390/a18010032
Mehmood RLazar MLiang XCorchado JSee S(2025)Editorial: Protecting privacy in neuroimaging analysis: balancing data sharing and privacy preservationFrontiers in Neuroinformatics10.3389/fninf.2024.154312118Online publication date: 7-Jan-2025
https://doi.org/10.3389/fninf.2024.1543121
Zhang XZhou YWu DSheng QRiaz SHu MXiao L(2025)A Survey on Privacy-Preserving Caching at Network Edge: Classification, Solutions, and ChallengesACM Computing Surveys10.1145/370663057:5(1-38)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3706630
Xin RWang JChen PZhao Z(2025)Trustworthy AI-based Performance Diagnosis Systems for Cloud Applications: A ReviewACM Computing Surveys10.1145/370174057:5(1-37)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3701740
Ying CXia FWei DYu XXu YZhang WJiang XJin HLuo YZhang TTao D(2025)BIT-FL: Blockchain-Enabled Incentivized and Secure Federated Learning FrameworkIEEE Transactions on Mobile Computing10.1109/TMC.2024.347761624:2(1212-1229)Online publication date: Feb-2025
https://doi.org/10.1109/TMC.2024.3477616
Yan JZheng YYang XChen CGuan X(2025)Privacy-Preserving Localization for Underwater Acoustic Sensor Networks: A Differential Privacy-Based Deep Learning ApproachIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.351806920(737-752)Online publication date: 2025
https://doi.org/10.1109/TIFS.2024.3518069
Zhang SHuang JLi P(2025)Analyze and Improve Differentially Private Federated Learning: A Model Robustness PerspectiveIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.351805820(807-821)Online publication date: 2025
https://doi.org/10.1109/TIFS.2024.3518058
Wang NWang SLi MWu LZhang ZGuan ZZhu L(2025)Balancing Differential Privacy and Utility: A Relevance-Based Adaptive Private Fine-Tuning Framework for Language ModelsIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.351657920(207-220)Online publication date: 2025
https://doi.org/10.1109/TIFS.2024.3516579
Dai ZGao YZhou CFu AZhang ZXue MZheng YZhang Y(2025)Decaf: Data Distribution Decompose Attack Against Federated LearningIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.351654520(405-420)Online publication date: 2025
https://doi.org/10.1109/TIFS.2024.3516545
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents