Amartya Sanyal
LightNews LightNews
amartyasanyal.bsky.social
Amartya Sanyal
@amartyasanyal.bsky.social
960 followers 150 following 18 posts
Assistant Professor @Dept. Of Computer Science, University of Copenhagen, Ex Postdoc @MPI-IS, ETHZ, PhD @University of Oxford, B.Tech @CSE,IITK.
Posts Replies Media Videos
Reposted by Amartya Sanyal
omribendov.bsky.social
In our new work we ask: Can end-users make a platform’s ML models fairer?

Firm-side fair learning often reduces accuracy, discouraging firms from using it. But if a platform relies on user data, can minority users collectively change the data to induce fairness?

(1/4)
A classifier has an error of 0.15 and unfairness violation of 0.13, while the same classifier, on data with 6 relabeled samples, have the the same error but 0.03 fairness violation.
August 22, 2025 at 6:45 AM
Reposted by Amartya Sanyal
dppapers.bsky.social
An Iterative Algorithm for Differentially Private $k$-PCA with Adaptive Noise

Johanna Düngler, Amartya Sanyal

http://arxiv.org/abs/2508.10879
An Iterative Algorithm for Differentially Private $k$-PCA with Adaptive Noise Johanna Düngler, Amartya Sanyal http://arxiv.org/abs/2508.10879 Given $n$ i.i.d. random matrices $A_i \in \mathbb{R}^{d \times d}$ that share a common expectation $\Sigma$, the objective of Differentially Private Stochastic PCA is to identify a subspace of dimension $k$ that captures the largest variance directions of $\Sigma$, while preserving differential privacy (DP) of each individual $A_i$. Existing methods either (i) require the sample size $n$ to scale super-linearly with dimension $d$, even under Gaussian assumptions on the $A_i$, or (ii) introduce excessive noise for DP even when the intrinsic randomness within $A_i$ is small. Liu et al. (2022a) addressed these issues for sub-Gaussian data but only for estimating the top eigenvector ($k=1$) using their algorithm DP-PCA. We propose the first algorithm capable of estimating the top $k$ eigenvectors for arbitrary $k \leq d$, whilst overcoming both limitations above. For $k=1$ our algorithm matches the utility guarantees of DP-PCA, achieving near-optimal statistical error even when $n = \tilde{\!O}(d)$. We further provide a lower bound for general $k > 1$, matching our upper bound up to a factor of $k$, and experimentally demonstrate the advantages of our algorithm over comparable baselines.
August 15, 2025 at 3:50 AM
amartyasanyal.bsky.social
🚨New Paper: Online Learning and Unlearning 🚨

We look at learning and unlearning in the online setting where both learning and unlearning requests arrive continuously over time.

Lead by @yaxihu.bsky.social and joint work with Bernhard Schölkopf

arxiv.org/abs/2505.08557
May 15, 2025 at 3:46 PM
Reposted by Amartya Sanyal
ukplab.bsky.social
𝗖𝗮𝗻 𝘄𝗲 𝗮𝗹𝗶𝗴𝗻 𝗟𝗟𝗠𝘀 𝘁𝗼𝘄𝗮𝗿𝗱𝘀 𝗮 𝗱𝗲𝘀𝗶𝗿𝗲𝗱 𝗯𝗲𝗵𝗮𝘃𝗶𝗼𝘂𝗿 𝘄𝗵𝗶𝗹𝗲 𝗺𝗮𝗶𝗻𝘁𝗮𝗶𝗻𝗶𝗻𝗴 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁𝗶𝗮𝗹 𝗽𝗿𝗶𝘃𝗮𝗰𝘆 𝗴𝘂𝗮𝗿𝗮𝗻𝘁𝗲𝗲𝘀?
We answer this in our #ICLR2025 paper.

Tl;dr: We propose, evaluate and audit a novel differentially private activation steering algorithm for aligning LLMs.

(1/🧵)
April 23, 2025 at 9:15 AM
amartyasanyal.bsky.social
Open postdoc position in learning theory/ privacy/ robustness/ unlearning or any related topics with me and others in University of Copenhagen, Denmark.

If you think you would be a good candidate, send me an email amartya18x.github.io/hiring/

#postdoc
Vacancies
amartya18x.github.io
March 27, 2025 at 8:23 PM
Reposted by Amartya Sanyal
rieck.mlsec.org
No plans for April 9–11 yet? — Why not spend an amazing week in beautiful Copenhagen 🇩🇰, exploring cutting-edge research on trustworthy machine learning.

Join us at SaTML 2025, the premier conference on AI security, AI privacy, and AI fairness!

👉 satml.org/attend

@satml.org
IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), April 9-11, 2025, Copenhagen.
March 3, 2025 at 3:03 PM
amartyasanyal.bsky.social
Very shortly at @realaaai.bsky.social, @alext2.bsky.social and I will be giving a Tutorial on the impact of Quality and availability of labels and data for Privacy, Fairness, and Robustness of ML algorithms

See here amartya18x.github.io/files/Tutori...

@ucph.bsky.social
February 25, 2025 at 12:27 PM
Reposted by Amartya Sanyal
aicentre.dk
3rd IEEE Conference on Secure and Trustworthy Machine Learning
University of Copenhagen, Denmark, April 9-11, 2025 - registration is open. satml.org

@amartyasanyal.bsky.social
IEEE SaTML
IEEE Conference on Secure and Trustworthy Machine Learning
satml.org
February 22, 2025 at 4:43 PM
Reposted by Amartya Sanyal
dppapers.bsky.social
Differentially Private Steering for Large Language Model Alignment
Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal
http://arxiv.org/abs/2501.18532
Differentially Private Steering for Large Language Model Alignment Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal http://arxiv.org/abs/2501.18532 Aligning Large Language Models (LLMs) with human values and away from undesirable behaviors (such as hallucination) has become increasingly important. Recently, steering LLMs towards a desired behavior via activation editing has emerged as an effective method to mitigate harmful generations at inference-time. Activation editing modifies LLM representations by preserving information from positive demonstrations (e.g., truthful) and minimising information from negative demonstrations (e.g., hallucinations). When these demonstrations come from a private dataset, the aligned LLM may leak private information contained in those private samples. In this work, we present the first study of aligning LLM behavior with private datasets. Our work proposes the \textit{\underline{P}rivate \underline{S}teering for LLM \underline{A}lignment (PSA)} algorithm to edit LLM activations with differential privacy (DP) guarantees. We conduct extensive experiments on seven different benchmarks with open-source LLMs of different sizes (0.5B to 7B) and model families (LlaMa, Qwen, Mistral and Gemma). Our results show that PSA achieves DP guarantees for LLM alignment with minimal loss in performance, including alignment metrics, open-ended text generation quality, and general-purpose reasoning. We also develop the first Membership Inference Attack (MIA) for evaluating and auditing the empirical privacy for the problem of LLM steering via activation editing. Our attack is tailored for activation editing and relies solely on the generated texts without their associated probabilities. Our experiments support the theoretical guarantees by showing improved guarantees for our \textit{PSA} algorithm compared to several existing non-private techniques.
January 31, 2025 at 4:33 AM
amartyasanyal.bsky.social
PhD call in Denmark. Applications open!
DDSA PhD Fellowship Call 2025 | DDSA
ddsa.dk
January 28, 2025 at 12:53 PM
Reposted by Amartya Sanyal
ukplab.bsky.social
#ICLR
»Differentially Private Steering for Large Language Model Alignment« by @anmolgoel.bsky.social, Yaxi Hu, Iryna Gurevych (@igurevych.bsky.social) & Amartya Sanyal (@amartyasanyal.bsky.social)

(2/🧵)
January 27, 2025 at 11:03 AM
amartyasanyal.bsky.social
I was very lucky and happy to be awarded the Villum Young Investigator grant yesterday villumfonden.dk/en/news/mill...

Looking forward to the resulting research in unlearning, privacy, and online learning supported by the Villum foundation.

(Hiring motivated PhDs and postdocs, especially postdocs)
Millions in funding for young researchers
Nineteen promising researchers working within the technical and natural sciences have received funding of DKK 150 million for their research projects.
villumfonden.dk
January 24, 2025 at 11:12 PM
Reposted by Amartya Sanyal
mahdihaghifam.bsky.social
I’ll be at #NeurIPS2024 this week! Looking forward to presenting my joint work with Thomas Steinke(@stein.ke) and Jon Ullman(@thejonullman.bsky.social)

NeurIPS page with video: neurips.cc/virtual/2024...

Link to arxiv: arxiv.org/abs/2406.07407
December 11, 2024 at 12:22 PM
Reposted by Amartya Sanyal
ddmitriev.bsky.social
Excited to present at #NeurIPS2024 our work on robust mixture learning!

How hard is mixture learning when (a lot of) outliers are present? We show that it's easier than it seems!

Join us at the poster session (Wed, 16:30 PT, West Ballroom A-D #5710).
December 10, 2024 at 8:31 PM
Reposted by Amartya Sanyal
rasmuspagh.net
Graduating with a PhD related to privacy and robustness in machine learning? Apply to this post-doc opening by @amartyasanyal.bsky.social: employment.ku.dk/faculty/?sho...
Postdoc in Privacy and Robustness of Machine Learning Algorithms
employment.ku.dk
December 3, 2024 at 9:57 AM
amartyasanyal.bsky.social
Two weeks remaining to apply to this position.

I'll also be at NeurIPS, if you want to chat you can DM or email me. :)
amartyasanyal.bsky.social Amartya Sanyal @amartyasanyal.bsky.social · Nov 19
Open Postdoctoral position in Privacy, unlearning, and Robustness in Machine Learning in University of Copenhagen to work with me!

Deadline: December 15th

If you want to spend a couple of years working on these exciting topics in beautiful Copenhagen, Apply here employment.ku.dk/all-vacancies/…
December 2, 2024 at 10:05 AM
amartyasanyal.bsky.social
Three long research meetings throughout the day with four brilliant collaborators.

It was a good day.
November 27, 2024 at 6:46 PM
Reposted by Amartya Sanyal
ccanonne.github.io
On the TCS job market: Hao Wu!

Hao Wu's research interests focus on both the theoretical and practical aspects of differentially private data analysis. He is actively pursuing opportunities in academia and industry.

1/2 #TCSSky #AcademicJobMarket
November 25, 2024 at 8:32 PM
Reposted by Amartya Sanyal
nandofioretto.bsky.social
🆘Help needed!

Are you working on Privacy (from a Technical (e.g., Differential Privacy), Policy, or Law perspective)?

Please give your availability to review for PPAI (ppai-workshop.github.io) if you can!
We'd highly appreciate it! 🙏

forms.gle/dqjVsBsR2y81...
Sixth AAAI Workshop on Privacy-Preserving Artificial Intelligence
ppai-workshop.github.io
November 23, 2024 at 12:47 AM
amartyasanyal.bsky.social
I made a starter pack for european researchers interested in some aspects of learning theory. The list is clearly inexhaustive. So please enter your suggestions in comments.

go.bsky.app/5o5uVnr
November 21, 2024 at 10:31 AM
amartyasanyal.bsky.social
Open Postdoctoral position in Privacy, unlearning, and Robustness in Machine Learning in University of Copenhagen to work with me!

Deadline: December 15th

If you want to spend a couple of years working on these exciting topics in beautiful Copenhagen, Apply here employment.ku.dk/all-vacancies/…
November 19, 2024 at 8:29 AM