Anshuman Suri
@iamgroot42.bsky.social
Postdoc @ Khoury | Previously Ph.D. @ UVA (David Evans) | IIITD Alum | Interested in machine learning privacy & security.
1/ Most membership inference attacks (MIAs) have seemingly converged to black-box settings, driven by empirical evidence and theoretical folklore suggesting black-box access was optimal. But what if this assumption missed something critical? 😨
tl;dr? It did 🧵
tl;dr? It did 🧵
December 18, 2024 at 3:39 AM
1/ Most membership inference attacks (MIAs) have seemingly converged to black-box settings, driven by empirical evidence and theoretical folklore suggesting black-box access was optimal. But what if this assumption missed something critical? 😨
tl;dr? It did 🧵
tl;dr? It did 🧵
Temporally shifted data splits in membership inference can be misleading ⚠️ Be cautious when interpreting these benchmarks!
1/6 A lot of us are grappling with peer review these days, but its worst manifestation is when prestigious conference awards overlook critical flaws.
Case in point: #EMNLP2024 ’s Best Paper Award.
I & @iamgroot42.bsky.social wrote a blog on what went wrong: www.anshumansuri.com/blog/2024/ca... 🧵
Case in point: #EMNLP2024 ’s Best Paper Award.
I & @iamgroot42.bsky.social wrote a blog on what went wrong: www.anshumansuri.com/blog/2024/ca... 🧵
November 26, 2024 at 6:17 PM
Temporally shifted data splits in membership inference can be misleading ⚠️ Be cautious when interpreting these benchmarks!