| title | Exploring The Empirical Privacy Landscape of Synthetic Data |
|---|---|
| authors | Zihao Zhao, Avi Schwarzschild, Casey Meehan, Katherine Lee, Zhili Feng, Dylan Sam |
| date | 2026-10-15 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.22659 |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Methods for training language models on sensitive data differ in how they balance capability gains against the risk of leaking private information. In this paper, we present a controlled comparison of data-level and training-level interventions for navigating the capability-privacy tradeoff on synthetic medical records with canaries to probe for leakage. Direct finetuning performs best on capability metrics but leads to the most privacy leakage, while DP-SGD reduces leakage at substantial cost to capabilities. Generating synthetic data from these records offers a more favorable balance, particularly when guided by a uniqueness-aware instruction to replace private details. Relative to naive paraphrasing, privacy-aware rewriting improves privacy protection across a variety of different rewriting models. We also find that more capable rewriting models generally reduce the risks of downstream privacy leakage, although held-out fit does not improve monotonically and increased test-time compute yields inconsistent gains. Overall, these results highlight privacy-aware rewriting as a promising direction for achieving a more favorable trade-off between capabilities and empirical privacy compared to other training-level methods, and our evaluation framework supports comparisons of future models and methods. |
cite
@misc{zhao2026exploring,
title={Exploring The Empirical Privacy Landscape of Synthetic Data},
author={Zihao Zhao and Avi Schwarzschild and Casey Meehan and Katherine Lee and Zhili Feng and Dylan Sam},
year={2026},
eprint={2610.22659},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.22659}
}