Differentially Private Image Classification by Learning Priors from Random Processes
In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating pri...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Labor Dynamics Institute
2025-03-01
|
| Series: | The Journal of Privacy and Confidentiality |
| Subjects: | |
| Online Access: | https://journalprivacyconfidentiality.org/index.php/jpc/article/view/910 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849731381927608320 |
|---|---|
| author | Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal |
| author_facet | Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal |
| author_sort | Xinyu Tang |
| collection | DOAJ |
| description |
In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\epsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\epsilon=1$.
|
| format | Article |
| id | doaj-art-4bbb39ea350043a483a33c04af07e314 |
| institution | DOAJ |
| issn | 2575-8527 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Labor Dynamics Institute |
| record_format | Article |
| series | The Journal of Privacy and Confidentiality |
| spelling | doaj-art-4bbb39ea350043a483a33c04af07e3142025-08-20T03:08:35ZengLabor Dynamics InstituteThe Journal of Privacy and Confidentiality2575-85272025-03-0115110.29012/jpc.910Differentially Private Image Classification by Learning Priors from Random ProcessesXinyu Tang0Ashwinee Panda1Vikash Sehwag2Prateek Mittal3Princeton UniversityPrinceton UniversityPrinceton UniversityPrinceton University In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\epsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\epsilon=1$. https://journalprivacyconfidentiality.org/index.php/jpc/article/view/910differential privacyimage classification |
| spellingShingle | Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal Differentially Private Image Classification by Learning Priors from Random Processes The Journal of Privacy and Confidentiality differential privacy image classification |
| title | Differentially Private Image Classification by Learning Priors from Random Processes |
| title_full | Differentially Private Image Classification by Learning Priors from Random Processes |
| title_fullStr | Differentially Private Image Classification by Learning Priors from Random Processes |
| title_full_unstemmed | Differentially Private Image Classification by Learning Priors from Random Processes |
| title_short | Differentially Private Image Classification by Learning Priors from Random Processes |
| title_sort | differentially private image classification by learning priors from random processes |
| topic | differential privacy image classification |
| url | https://journalprivacyconfidentiality.org/index.php/jpc/article/view/910 |
| work_keys_str_mv | AT xinyutang differentiallyprivateimageclassificationbylearningpriorsfromrandomprocesses AT ashwineepanda differentiallyprivateimageclassificationbylearningpriorsfromrandomprocesses AT vikashsehwag differentiallyprivateimageclassificationbylearningpriorsfromrandomprocesses AT prateekmittal differentiallyprivateimageclassificationbylearningpriorsfromrandomprocesses |