Anupama Sridhar
anusridhar.bsky.social
Anupama Sridhar
@anusridhar.bsky.social
Professional calculator and non-smooth operator: RL theory, optimizer dynamics. All cat analogies here are mine.
Reposted by Anupama Sridhar
Anupama Sridhar, Alexander Johansen
Convergence of Adam in Deep ReLU Networks via Directional Complexity and Kakeya Bounds
https://arxiv.org/abs/2505.15013
May 22, 2025 at 5:41 AM
td(0) with nonlinear function approximation for deepnets
not in the lazy regime
not assuming your data is nice
not even pretending initialization matters

mean-field, mixing chains, convergence proofs
paper not out yet. somebody pray for the reviewers
#mathviolence
a cartoon drawing of a man 's head with a loading bar in front of a green board with math equations
ALT: a cartoon drawing of a man 's head with a loading bar in front of a green board with math equations
media.tenor.com
May 21, 2025 at 10:48 PM
Me: Self-taught advanced stats, probability, algebraic topology, differential geometry, stochastic processes, etc.
Also me: Fails Twitter’s “select all the bicycles” test three times in a row.
Maybe I am the stochastic process.

#ImNotARobotButIDefineOne
May 20, 2025 at 7:45 PM
Me: I'm in ML!
Them: So you make AI?
Me: No, I prove why it maybe works under very specific conditions.
Them: Sounds useful.
Me: I promise it is. To like... six people.
May 20, 2025 at 7:41 PM
Bound proving is 10% math, 90% convincing your coauthor the constant doesn’t matter that much.
May 20, 2025 at 7:40 PM
Proving bounds is just competitive overthinking on LaTeX. You stare at a matrix norm for 6 hours, take a walk, and suddenly remember a lemma from 2012 that saves 3 constants.
#MLTheory
May 20, 2025 at 7:40 PM
A Kakeya set is the smallest space a cat can spin in every direction.

That’s your ReLU network.
Track those spins, and you get tighter control than PAC-Bayes.

Cats don’t take random walks. Neither should your optimizer.

#CatsOfML #Kakeya #CSTheory
May 20, 2025 at 7:24 PM
PAC-Bayes tells you where the cat’s been.
Kakeya tells you where it can go.
We use cone crossings and directional bounds to track optimizer paths. Tighter than PAC-Bayes. No flat priors.
Wanna know how?
New ADAM paper drops soon.

#CatsOfML #Kakeya #OptimizerTheory #CSTheory
May 20, 2025 at 7:12 PM
Most TD(0) papers: keep the cat in a box, assume iid data, tiny steps, resets, data dropping, etc.
Ours: let the cat out. Nonlinear approximation, dependent data, real-world dynamics and it still finds the value function.
arxiv.org/pdf/2502.05706
#reinforcementlearning #catsofML #TDzero
arxiv.org
May 20, 2025 at 7:07 PM