Differentially Private Mixing of Public Datasets Improves Private Learning

Excerpt

arXiv:2610.06636v1 Announce Type: cross Abstract: Many machine learning applications involve sensitive data and therefore require training under differential privacy (DP). However, DP training often degrades model utility. In some cases, first pre-training the model on "public" data before finetuning with DP on the sensitive data can reduce the drop in utility. However, the success of this depends on how relevant the selected public dataset is to the sensitive data. We introduce the first pipeli