Petar Veličković
@petar-v.bsky.social
Senior Staff Research Scientist, Google DeepMind
Affiliated Lecturer, University of Cambridge
Associate, Clare Hall
GDL Scholar, ELLIS @ellis.eu
🇷🇸🇲🇪🇧🇦
Affiliated Lecturer, University of Cambridge
Associate, Clare Hall
GDL Scholar, ELLIS @ellis.eu
🇷🇸🇲🇪🇧🇦
The first draft 'G' chapter of the geometric deep learning book is live! 🚀
Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!)
I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎
Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!)
I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎
June 20, 2025 at 5:05 PM
The first draft 'G' chapter of the geometric deep learning book is live! 🚀
Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!)
I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎
Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!)
I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎
4yrs ago, the Geometric Deep Learning proto-book hit the arXiv 📚
Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀
Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢
@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀
Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢
@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
April 29, 2025 at 12:01 AM
4yrs ago, the Geometric Deep Learning proto-book hit the arXiv 📚
Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀
Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢
@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀
Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢
@mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
The recording of my 'LLMs as GNNs' talk is now public (link in thread) 💬🕸️
Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟
(And I look forward to refining this content even further -- especially as new works come trickling in 👀)
Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟
(And I look forward to refining this content even further -- especially as new works come trickling in 👀)
March 31, 2025 at 4:07 PM
The recording of my 'LLMs as GNNs' talk is now public (link in thread) 💬🕸️
Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟
(And I look forward to refining this content even further -- especially as new works come trickling in 👀)
Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟
(And I look forward to refining this content even further -- especially as new works come trickling in 👀)
This is happening today! 🌟
Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️
Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️
March 26, 2025 at 9:40 AM
This is happening today! 🌟
Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️
Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️
Reposted by Petar Veličković
GLOW is returning on 𝗠𝗮𝗿𝗰𝗵 𝟮𝟲𝘁𝗵, 𝟱𝗽𝗺 𝗖𝗘𝗧 with a special guest: @petar-v.bsky.social 🌟
He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session.
Specifically, we will learn how Graph ML tools can help understand LLM generalisation
He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session.
Specifically, we will learn how Graph ML tools can help understand LLM generalisation
March 20, 2025 at 7:39 PM
GLOW is returning on 𝗠𝗮𝗿𝗰𝗵 𝟮𝟲𝘁𝗵, 𝟱𝗽𝗺 𝗖𝗘𝗧 with a special guest: @petar-v.bsky.social 🌟
He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session.
Specifically, we will learn how Graph ML tools can help understand LLM generalisation
He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session.
Specifically, we will learn how Graph ML tools can help understand LLM generalisation
I'm excited to share that we'll have Ilan Price giving a talk at the University of Cambridge on GenCast -- a state-of-the-art model for probabilistic weather forecasting 🌦️⛈️⛅️
If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all!
All details (+ Zoom link) below!
If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all!
All details (+ Zoom link) below!
February 14, 2025 at 1:26 PM
I'm excited to share that we'll have Ilan Price giving a talk at the University of Cambridge on GenCast -- a state-of-the-art model for probabilistic weather forecasting 🌦️⛈️⛅️
If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all!
All details (+ Zoom link) below!
If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all!
All details (+ Zoom link) below!
And so we set out to understand _feedforward_ graphs (i.e. graphs w/o back edges) ⏩
Turns out these graphs are rather understudied for how often they are used in practice! 😮
We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀
Turns out these graphs are rather understudied for how often they are used in practice! 😮
We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀
February 11, 2025 at 10:12 AM
And so we set out to understand _feedforward_ graphs (i.e. graphs w/o back edges) ⏩
Turns out these graphs are rather understudied for how often they are used in practice! 😮
We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀
Turns out these graphs are rather understudied for how often they are used in practice! 😮
We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀
Reposted by Petar Veličković
Trying to copy the greats Pietro and @petar-v.bsky.social giving a guest lecture for the GDL class today!
February 10, 2025 at 9:25 PM
Trying to copy the greats Pietro and @petar-v.bsky.social giving a guest lecture for the GDL class today!
Reposted by Petar Veličković
*Round and Round We Go! What makes Rotary Positional Encodings useful?*
by @petar-v.bsky.social et al.
They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics.
arxiv.org/abs/2410.06205
by @petar-v.bsky.social et al.
They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics.
arxiv.org/abs/2410.06205
February 10, 2025 at 11:23 AM
*Round and Round We Go! What makes Rotary Positional Encodings useful?*
by @petar-v.bsky.social et al.
They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics.
arxiv.org/abs/2410.06205
by @petar-v.bsky.social et al.
They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics.
arxiv.org/abs/2410.06205
This summer, a group of elite AI scientists and engineers will come over to Sarajevo 🇧🇦 for #EEML2025 ❤️
We've been working hard behind the scenes to prepare a stellar program for our attendees!
Trust me, you don't want to miss this 🚀
Applications open now! (details in post)
We've been working hard behind the scenes to prepare a stellar program for our attendees!
Trust me, you don't want to miss this 🚀
Applications open now! (details in post)
Applications are now open for EEML 2025 in Sarajevo, Bosnia and Herzegovina, 21-26 July! 🎉
Learn from top AI researchers and connect with peers in Sarajevo 🇧🇦, a historical crossroads of East and West. Needs-based scholarships are available.
Deadline: 31 March 2025.
Learn from top AI researchers and connect with peers in Sarajevo 🇧🇦, a historical crossroads of East and West. Needs-based scholarships are available.
Deadline: 31 March 2025.
February 5, 2025 at 3:39 PM
This summer, a group of elite AI scientists and engineers will come over to Sarajevo 🇧🇦 for #EEML2025 ❤️
We've been working hard behind the scenes to prepare a stellar program for our attendees!
Trust me, you don't want to miss this 🚀
Applications open now! (details in post)
We've been working hard behind the scenes to prepare a stellar program for our attendees!
Trust me, you don't want to miss this 🚀
Applications open now! (details in post)
Exciting news: the NeurIPS Sci4DL Workshop has recognised our softmax paper as a best paper runner-up for its 'Debunking Challenge'! 🚀🧑🔬
I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌
I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌
December 16, 2024 at 10:23 AM
Exciting news: the NeurIPS Sci4DL Workshop has recognised our softmax paper as a best paper runner-up for its 'Debunking Challenge'! 🚀🧑🔬
I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌
I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌
It's official! 🎉 I am excited to share that EEML will be coming to Sarajevo 🇧🇦 next summer!
For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!).
Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀
For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!).
Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀
EEML'25, our yearly machine learning summer school event, will be organised next summer in the beautiful city of Sarajevo - the place where East meets West 🇧🇦🇧🇦🇧🇦.
More details coming soon, please see the link in the thread!
More details coming soon, please see the link in the thread!
December 15, 2024 at 6:28 PM
It's official! 🎉 I am excited to share that EEML will be coming to Sarajevo 🇧🇦 next summer!
For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!).
Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀
For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!).
Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀
Today it is time to present our softmax paper to the world! 🌡️
If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦
🧑🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am
🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am
If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦
🧑🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am
🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am
Just a poster I'm very proud of. Mainly because it's the first time I led a theory-focused paper 🥳
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
December 15, 2024 at 3:46 PM
Today it is time to present our softmax paper to the world! 🌡️
If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦
🧑🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am
🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am
If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦
🧑🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am
🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am
A very nice blog from Przemek Pietrzkiewicz, offering thoughts on our recent result in AI for competitive programming 🏆
Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀
Worth a read if you want to understand implications of our work! Link below ⬇️
Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀
Worth a read if you want to understand implications of our work! Link below ⬇️
December 8, 2024 at 12:34 PM
A very nice blog from Przemek Pietrzkiewicz, offering thoughts on our recent result in AI for competitive programming 🏆
Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀
Worth a read if you want to understand implications of our work! Link below ⬇️
Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀
Worth a read if you want to understand implications of our work! Link below ⬇️
Reposted by Petar Veličković
If you will be at #NeurIPS2024 @neuripsconf.bsky.social and would like to come see our models in action, come say hi 👋 and check out our demo at the GDM booth!
Wednesday, Dec. 11th @ 9:30-10:00.
Lots of other great things to see as well! Check it out: 👇
deepmind.google/discover/blo...
Wednesday, Dec. 11th @ 9:30-10:00.
Lots of other great things to see as well! Check it out: 👇
deepmind.google/discover/blo...
December 6, 2024 at 12:42 PM
If you will be at #NeurIPS2024 @neuripsconf.bsky.social and would like to come see our models in action, come say hi 👋 and check out our demo at the GDM booth!
Wednesday, Dec. 11th @ 9:30-10:00.
Lots of other great things to see as well! Check it out: 👇
deepmind.google/discover/blo...
Wednesday, Dec. 11th @ 9:30-10:00.
Lots of other great things to see as well! Check it out: 👇
deepmind.google/discover/blo...
Our team at Google DeepMind is hiring Student Researchers for 2025!
🧑🔬 Interested in understanding reasoning capabilities of neural networks from first principles?
🧑🎓 Currently studying for a BS/MS/PhD?
🧑💻 Have solid engineering and research skills?
🌟 We want to hear from you! Details in thread.
🧑🔬 Interested in understanding reasoning capabilities of neural networks from first principles?
🧑🎓 Currently studying for a BS/MS/PhD?
🧑💻 Have solid engineering and research skills?
🌟 We want to hear from you! Details in thread.
December 5, 2024 at 11:08 PM
Our team at Google DeepMind is hiring Student Researchers for 2025!
🧑🔬 Interested in understanding reasoning capabilities of neural networks from first principles?
🧑🎓 Currently studying for a BS/MS/PhD?
🧑💻 Have solid engineering and research skills?
🌟 We want to hear from you! Details in thread.
🧑🔬 Interested in understanding reasoning capabilities of neural networks from first principles?
🧑🎓 Currently studying for a BS/MS/PhD?
🧑💻 Have solid engineering and research skills?
🌟 We want to hear from you! Details in thread.
Reposted by Petar Veličković
I'm excited to share a new paper: "Mastering Board Games by External and Internal Planning with Language Models"
storage.googleapis.com/deepmind-med...
(also soon to be up on Arxiv, once it's been processed there)
storage.googleapis.com/deepmind-med...
(also soon to be up on Arxiv, once it's been processed there)
storage.googleapis.com
December 5, 2024 at 7:49 AM
I'm excited to share a new paper: "Mastering Board Games by External and Internal Planning with Language Models"
storage.googleapis.com/deepmind-med...
(also soon to be up on Arxiv, once it's been processed there)
storage.googleapis.com/deepmind-med...
(also soon to be up on Arxiv, once it's been processed there)
Reposted by Petar Veličković
Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.
December 4, 2024 at 4:01 PM
Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.
Just a poster I'm very proud of. Mainly because it's the first time I led a theory-focused paper 🥳
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
December 4, 2024 at 5:13 PM
Just a poster I'm very proud of. Mainly because it's the first time I led a theory-focused paper 🥳
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦
I unfortunately won't be there myself, but Christos and Federico will be around 🚀
A clear step towards achieving my dream: building AI that assists competitive programmers 🧑💻
“This is an exciting approach to combine work of human competitive programmers and LLMs, to achieve results that neither would achieve on their own.” --Petr Mitrichev
More details below! 🧵
“This is an exciting approach to combine work of human competitive programmers and LLMs, to achieve results that neither would achieve on their own.” --Petr Mitrichev
More details below! 🧵
December 2, 2024 at 11:46 AM
A clear step towards achieving my dream: building AI that assists competitive programmers 🧑💻
“This is an exciting approach to combine work of human competitive programmers and LLMs, to achieve results that neither would achieve on their own.” --Petr Mitrichev
More details below! 🧵
“This is an exciting approach to combine work of human competitive programmers and LLMs, to achieve results that neither would achieve on their own.” --Petr Mitrichev
More details below! 🧵
The key productivity hack I unlocked in recent years was accepting and being cognisant of the fact that I likely won't finish everything I was planning to do at the start of the day. 😶
Ironically, this led to ~2x more things being finished, because I spend less time worrying about falling behind 🥳
Ironically, this led to ~2x more things being finished, because I spend less time worrying about falling behind 🥳
November 27, 2024 at 8:21 PM
The key productivity hack I unlocked in recent years was accepting and being cognisant of the fact that I likely won't finish everything I was planning to do at the start of the day. 😶
Ironically, this led to ~2x more things being finished, because I spend less time worrying about falling behind 🥳
Ironically, this led to ~2x more things being finished, because I spend less time worrying about falling behind 🥳
Reposted by Petar Veličković
✨EXCITING NEWS! We now have a Bluesky account!!
AND registration for the 3rd Learning on Graphs conference is open 😊
It is virtual, free to attend, livestreamed, and recorded 📹 Sign up today! The conference is this coming week 😱
forms.gle/eYiDCopJGUc8...
AND registration for the 3rd Learning on Graphs conference is open 😊
It is virtual, free to attend, livestreamed, and recorded 📹 Sign up today! The conference is this coming week 😱
forms.gle/eYiDCopJGUc8...
November 24, 2024 at 5:00 PM
✨EXCITING NEWS! We now have a Bluesky account!!
AND registration for the 3rd Learning on Graphs conference is open 😊
It is virtual, free to attend, livestreamed, and recorded 📹 Sign up today! The conference is this coming week 😱
forms.gle/eYiDCopJGUc8...
AND registration for the 3rd Learning on Graphs conference is open 😊
It is virtual, free to attend, livestreamed, and recorded 📹 Sign up today! The conference is this coming week 😱
forms.gle/eYiDCopJGUc8...
As an example from our recent paper, turns out that assuming a _tokenised vocabulary_ input allows one to apply mathematical analysis results due to Weierstrass, which in turn implies that all sharp attention heads must collapse.
Crazy chain of thought, and not at all what we expected at the start.
Crazy chain of thought, and not at all what we expected at the start.
My fav theoretical proofs in AI papers tend to be ones that make assumptions that:
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
November 23, 2024 at 3:22 PM
As an example from our recent paper, turns out that assuming a _tokenised vocabulary_ input allows one to apply mathematical analysis results due to Weierstrass, which in turn implies that all sharp attention heads must collapse.
Crazy chain of thought, and not at all what we expected at the start.
Crazy chain of thought, and not at all what we expected at the start.
My fav theoretical proofs in AI papers tend to be ones that make assumptions that:
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
November 23, 2024 at 2:48 PM
My fav theoretical proofs in AI papers tend to be ones that make assumptions that:
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
* realistically conform to the 'situation on the ground', i.e. ~match models used in practice;
* completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.