Max Zhdanov
@maxxxzdn.bsky.social
PhD candidate at AMLab with Max Welling and Jan-Willem van de Meent.
Research in physics-inspired and geometric deep learning.
Research in physics-inspired and geometric deep learning.
🤹 New blog post!
I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025
blog: maxxxzdn.github.io/blog/erwin/
paper: arxiv.org/abs/2502.17019
Compressed version in the thread below:
I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025
blog: maxxxzdn.github.io/blog/erwin/
paper: arxiv.org/abs/2502.17019
Compressed version in the thread below:
June 21, 2025 at 4:23 PM
🤹 New blog post!
I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025
blog: maxxxzdn.github.io/blog/erwin/
paper: arxiv.org/abs/2502.17019
Compressed version in the thread below:
I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025
blog: maxxxzdn.github.io/blog/erwin/
paper: arxiv.org/abs/2502.17019
Compressed version in the thread below:
Can only speak for my ICML reviewing batch, but the hack of putting scary, convoluted and wrong math still works.
March 25, 2025 at 7:13 AM
Can only speak for my ICML reviewing batch, but the hack of putting scary, convoluted and wrong math still works.
🤹 Excited to share Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems
joint work with @wellingmax.bsky.social and @jwvdm.bsky.social
preprint: arxiv.org/abs/2502.17019
code: github.com/maxxxzdn/erwin
joint work with @wellingmax.bsky.social and @jwvdm.bsky.social
preprint: arxiv.org/abs/2502.17019
code: github.com/maxxxzdn/erwin
March 5, 2025 at 6:04 PM
🤹 Excited to share Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems
joint work with @wellingmax.bsky.social and @jwvdm.bsky.social
preprint: arxiv.org/abs/2502.17019
code: github.com/maxxxzdn/erwin
joint work with @wellingmax.bsky.social and @jwvdm.bsky.social
preprint: arxiv.org/abs/2502.17019
code: github.com/maxxxzdn/erwin
Interestingly, something similar can be done for irregular data structures - point clouds - with a couple of tricks including hierarchical trees and FlexAttention🌳
1/N 🧵
1/N 🧵
DeepSeek's NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference!
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
Core components of NSA:
• Dynamic hierarchical sparse strategy
• Coarse-grained token compression
• Fine-grained token selection
February 20, 2025 at 10:02 PM
Interestingly, something similar can be done for irregular data structures - point clouds - with a couple of tricks including hierarchical trees and FlexAttention🌳
1/N 🧵
1/N 🧵
the number of snarky comments about Musk from anime pfps on Twitter lately left me wondering how much of the right turn of the group is just teenage contrarianism
February 17, 2025 at 4:02 PM
the number of snarky comments about Musk from anime pfps on Twitter lately left me wondering how much of the right turn of the group is just teenage contrarianism
Reposted by Max Zhdanov
🚨 Excited to share our work just published in Physical Review Letters with @wellingmax.bsky.social, @jwvdm.bsky.social, @berndensing.bsky.social, Marjolein Dijkstra and René van Roij: doi.org/10.1103/Phys....
Details below 👇
Details below 👇
Learning Neural Free-Energy Functionals with Pair-Correlation Matching
The intrinsic Helmholtz free-energy functional, the centerpiece of classical density functional theory, is at best only known approximately for 3D systems. Here we introduce a method for learning a ne...
doi.org
February 13, 2025 at 9:21 AM
🚨 Excited to share our work just published in Physical Review Letters with @wellingmax.bsky.social, @jwvdm.bsky.social, @berndensing.bsky.social, Marjolein Dijkstra and René van Roij: doi.org/10.1103/Phys....
Details below 👇
Details below 👇
jax is very aesthetic until you need to do masked select
February 4, 2025 at 1:36 PM
jax is very aesthetic until you need to do masked select
This is how it should be: one file, ctrl+c ctrl+v runnable from a notebook. I really do not want to mess with omegaconf, hydra or, the worst, a custom configuration system just to figure out how your model runs.
January 26, 2025 at 11:17 AM
This is how it should be: one file, ctrl+c ctrl+v runnable from a notebook. I really do not want to mess with omegaconf, hydra or, the worst, a custom configuration system just to figure out how your model runs.
The comments section is depressing to read. I guess automation is inevitable in a progressing society, but such incidents do bring a bitter flavour to the current advancements.
Happy 2025. I Lost My Freelance Writing Job To AI.
YouTube video by Alex Wei
www.youtube.com
January 8, 2025 at 5:51 PM
The comments section is depressing to read. I guess automation is inevitable in a progressing society, but such incidents do bring a bitter flavour to the current advancements.
grabbing the whole node on a cluster for an interactive session for a week should be considered antisocial behavior
January 7, 2025 at 8:45 AM
grabbing the whole node on a cluster for an interactive session for a week should be considered antisocial behavior
Reposted by Max Zhdanov
🚨 FINAL REMINDER 🚨: Multiple Postdoc and PhD positions in our AI + Physics cluster in DAMTP, Cambridge!
Deadlines:
- Postdoc: Jan 5th (Sunday)
- PhD: Jan 7th (Tuesday)
More info below – Please share with researchers and students who might be interested in joining us!
Deadlines:
- Postdoc: Jan 5th (Sunday)
- PhD: Jan 7th (Tuesday)
More info below – Please share with researchers and students who might be interested in joining us!
January 2, 2025 at 6:32 PM
🚨 FINAL REMINDER 🚨: Multiple Postdoc and PhD positions in our AI + Physics cluster in DAMTP, Cambridge!
Deadlines:
- Postdoc: Jan 5th (Sunday)
- PhD: Jan 7th (Tuesday)
More info below – Please share with researchers and students who might be interested in joining us!
Deadlines:
- Postdoc: Jan 5th (Sunday)
- PhD: Jan 7th (Tuesday)
More info below – Please share with researchers and students who might be interested in joining us!
tired: graphs
wired: trees
wired: trees
December 29, 2024 at 9:41 AM
tired: graphs
wired: trees
wired: trees
Best lab account around here
The Christmas spirit has arrived at AMLab! 🎄✨
Yesterday we kicked off the holidays with a festive group dinner and a fun Secret Santa exchange. 🎅
Wishing everyone a restful and joyful winter break and a happy new year! ❄️💫
Yesterday we kicked off the holidays with a festive group dinner and a fun Secret Santa exchange. 🎅
Wishing everyone a restful and joyful winter break and a happy new year! ❄️💫
December 20, 2024 at 10:57 AM
Best lab account around here
Reposted by Max Zhdanov
🚨 PhD position alert! 🚨
I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...
I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...
PhD Position in Mechanistic Interpretability
PhD Position in Mechanistic Interpretability
vacatures.uva.nl
December 2, 2024 at 7:36 PM
🚨 PhD position alert! 🚨
I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...
I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...
Reposted by Max Zhdanov
If you are attending #NeurIPS2024🇨🇦, make sure to check out AMLab's 11 accepted papers ...and to have a chat with our members there! 👩🔬🍻☕
Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list!
🧵1 / 12
Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list!
🧵1 / 12
December 9, 2024 at 1:24 PM
If you are attending #NeurIPS2024🇨🇦, make sure to check out AMLab's 11 accepted papers ...and to have a chat with our members there! 👩🔬🍻☕
Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list!
🧵1 / 12
Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list!
🧵1 / 12
Reposted by Max Zhdanov
I will be at NeurIPS this week. @amlab.bsky.social will be presenting 10 papers in total, and I myself am involved with 3 (see thread).
I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!
I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!
December 9, 2024 at 10:11 AM
I will be at NeurIPS this week. @amlab.bsky.social will be presenting 10 papers in total, and I myself am involved with 3 (see thread).
I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!
I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!
At a certain point, I was so inspired by the geomancer paper by @davidpfau.com et al. that I decided to apply for a PhD in causality/GDL at ELLIS. All those buzzwords in my motivational letter just to find later that I was writing about casual ML all along 🤦♂️
Search Google Scholar for "casual inference" (not causal inference).
December 6, 2024 at 1:01 PM
At a certain point, I was so inspired by the geomancer paper by @davidpfau.com et al. that I decided to apply for a PhD in causality/GDL at ELLIS. All those buzzwords in my motivational letter just to find later that I was writing about casual ML all along 🤦♂️
Reposted by Max Zhdanov
hi everyone!! let's try this optimal transport again 🙃
December 5, 2024 at 12:58 PM
hi everyone!! let's try this optimal transport again 🙃
Reposted by Max Zhdanov
Yesterday @ellisamsterdam.bsky.social hosted the yearly NeurIPS-Fest, a pre-party for NeurIPS with a keynote talk, poster session, drinks and bites! 🍺🍻
The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥
The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥
November 29, 2024 at 1:38 PM
Yesterday @ellisamsterdam.bsky.social hosted the yearly NeurIPS-Fest, a pre-party for NeurIPS with a keynote talk, poster session, drinks and bites! 🍺🍻
The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥
The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥
Reposted by Max Zhdanov
Fun paper led by Julia Balla: "A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing".
Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month.
📄: arxiv.org/abs/2410.20516
💻: github.com/smsharma/eqn...
Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month.
📄: arxiv.org/abs/2410.20516
💻: github.com/smsharma/eqn...
A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing
Efficiently processing structured point cloud data while preserving multiscale information is a key challenge across domains, from graphics to atomistic modeling. Using a curated dataset of simulated ...
arxiv.org
November 26, 2024 at 5:59 PM
Fun paper led by Julia Balla: "A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing".
Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month.
📄: arxiv.org/abs/2410.20516
💻: github.com/smsharma/eqn...
Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month.
📄: arxiv.org/abs/2410.20516
💻: github.com/smsharma/eqn...
Reposted by Max Zhdanov
Arvind, me, and Jonah released a new pre-print on some pen and paper analysis of fundamental failure modes and old school stability analysis for neural PDEs typically used in AI for Science application. arxiv.org/abs/2411.15101. 1/n
What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning
Differentiable Programming for scientific machine learning (SciML) has recently seen considerable interest and success, as it directly embeds neural networks inside PDEs, often called as NeuralPDEs, d...
arxiv.org
November 25, 2024 at 8:45 PM
Arvind, me, and Jonah released a new pre-print on some pen and paper analysis of fundamental failure modes and old school stability analysis for neural PDEs typically used in AI for Science application. arxiv.org/abs/2411.15101. 1/n
Reposted by Max Zhdanov
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
November 25, 2024 at 5:52 PM
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
Reposted by Max Zhdanov
Meet our Lab's members: staff, postdocs and PhD students! :)
With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋
go.bsky.app/8EGigUy
With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋
go.bsky.app/8EGigUy
November 21, 2024 at 9:22 PM
Meet our Lab's members: staff, postdocs and PhD students! :)
With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋
go.bsky.app/8EGigUy
With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋
go.bsky.app/8EGigUy
New year's resolution: make it to the grumpy ML starter pack
November 21, 2024 at 11:38 AM
New year's resolution: make it to the grumpy ML starter pack
Reposted by Max Zhdanov
Hi everyone! This is AMLab :)
Looking forward to share our research here on 🦋 !
Looking forward to share our research here on 🦋 !
November 19, 2024 at 4:00 PM
Hi everyone! This is AMLab :)
Looking forward to share our research here on 🦋 !
Looking forward to share our research here on 🦋 !