TR2021-082
Robust Machine Learning via Privacy/Rate-Distortion Theory
-
- "Robust Machine Learning via Privacy/Rate-Distortion Theory", IEEE International Symposium on Information Theory (ISIT), DOI: 10.1109/ISIT45174.2021.9517751, July 2021.BibTeX TR2021-082 PDF Video Presentation
- @inproceedings{Wang2021jul,
- author = {Wang, Ye and Aeron, Shuchin and Rakin, Adnan S and Koike-Akino, Toshiaki and Moulin, Pierre},
- title = {Robust Machine Learning via Privacy/Rate-Distortion Theory},
- booktitle = {IEEE International Symposium on Information Theory (ISIT)},
- year = 2021,
- month = jul,
- publisher = {IEEE},
- doi = {10.1109/ISIT45174.2021.9517751},
- isbn = {978-1-5386-8210-4},
- url = {https://www.merl.com/publications/TR2021-082}
- }
,
- "Robust Machine Learning via Privacy/Rate-Distortion Theory", IEEE International Symposium on Information Theory (ISIT), DOI: 10.1109/ISIT45174.2021.9517751, July 2021.
-
MERL Contacts:
-
Research Areas:
Artificial Intelligence, Machine Learning, Signal Processing
Abstract:
Robust machine learning formulations have emerged to address the prevalent vulnerability of deep neural networks to adversarial examples. Our work draws the connection between optimal robust learning and the privacy-utility tradeoff problem, which is a generalization of the rate-distortion problem. The saddle point of the game between a robust classifier and an adversarial perturbation can be found via the solution of a maximum conditional entropy problem. This information-theoretic perspective sheds light on the fundamental tradeoff between robustness and clean data performance, which ultimately arises from the geometric structure of the underlying data distribution and perturbation constraints.