Differential Privacy: Adding Noise to BERT Embeddings

Duration: Jan 2025 - April 2025
Organization: Coriolis Technologies

Project Overview

Lightweight, post-processing defenses to protect BERT embeddings from privacy leakage (embedding inversion attacks) without modifying the model architecture.

Project Details

Objective: Enhance privacy of pretrained embeddings while preserving utility.

Techniques:

• Post-facto noise (Gaussian, Laplace, Uniform)
• Random projection
• Random rotation
• Squaring select dimensions (sign-preserved)

Evaluation:

• Datasets: STS-B, QQP
• Metrics: cosine similarity deviation, token retrieval, attack success reduction

For in-depth methodology and findings, see the full report