Jesse Engel
@jesseengel.bsky.social
Guitarist, Researcher Google DeepMind. Opinions are my own.
Thanks again to @hollyherndon.bsky.social, it's always fun collaborating with you 🙏
August 8, 2025 at 1:31 AM
Thanks again to @hollyherndon.bsky.social, it's always fun collaborating with you 🙏
These latest features take us one step closer to live interactive generative music models that are open, personalizable, and run locally on-device, so you can really trust them to hack them like any other instrument.
Next step, live on-device inference, stay tuned… 😀
Next step, live on-device inference, stay tuned… 😀
August 8, 2025 at 1:31 AM
These latest features take us one step closer to live interactive generative music models that are open, personalizable, and run locally on-device, so you can really trust them to hack them like any other instrument.
Next step, live on-device inference, stay tuned… 😀
Next step, live on-device inference, stay tuned… 😀
As we’ve seen with Genie, real-time interactive generative models opens up new worlds, and we’re excited to keep pushing the boundaries of what’s possible.
August 8, 2025 at 1:31 AM
As we’ve seen with Genie, real-time interactive generative models opens up new worlds, and we’re excited to keep pushing the boundaries of what’s possible.
This new wave of features is the culmination of effort from many folks at Google DeepMind, on the Lyria team and beyond. We are grateful to work with so many amazing collaborators to build the next generation of human-centered music AI tools. 🙏
August 8, 2025 at 1:31 AM
This new wave of features is the culmination of effort from many folks at Google DeepMind, on the Lyria team and beyond. We are grateful to work with so many amazing collaborators to build the next generation of human-centered music AI tools. 🙏
We hope Magenta RT finetuning can serve that purpose, and are happy to have their model as a first example of personalizing live generative music.
August 8, 2025 at 1:31 AM
We hope Magenta RT finetuning can serve that purpose, and are happy to have their model as a first example of personalizing live generative music.
For years, Holly and Mat Dryhurst have pioneered artist-centric ways in which AI can be used to push the boundaries of expression, and how artists and communities can have more agency over developing models for their own purposes (as they open sourced her voice to holly.plus for experimentation).
Holly+
Holly Herndon's digital twin
holly.plus
August 8, 2025 at 1:31 AM
For years, Holly and Mat Dryhurst have pioneered artist-centric ways in which AI can be used to push the boundaries of expression, and how artists and communities can have more agency over developing models for their own purposes (as they open sourced her voice to holly.plus for experimentation).
We're thrilled to have worked with Holly Herndon to train a model finetuned on her vocal style to share with all of you.
Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice
Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice
August 8, 2025 at 1:31 AM
We're thrilled to have worked with Holly Herndon to train a model finetuned on her vocal style to share with all of you.
Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice
Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice
We’re also excited to add support for artists to customize Magenta RT on their own training data!
g.co/magenta/rt-finetuning-colab
g.co/magenta/rt-finetuning-colab
August 8, 2025 at 1:31 AM
We’re also excited to add support for artists to customize Magenta RT on their own training data!
g.co/magenta/rt-finetuning-colab
g.co/magenta/rt-finetuning-colab
Audio injection also supports *live* music input. You can jam along with it and use it as an intelligent live looper!
Try it out for yourself on free Colab TPUs!
g.co/magenta/rt-audio-injection-colab
Try it out for yourself on free Colab TPUs!
g.co/magenta/rt-audio-injection-colab
August 8, 2025 at 1:31 AM
Audio injection also supports *live* music input. You can jam along with it and use it as an intelligent live looper!
Try it out for yourself on free Colab TPUs!
g.co/magenta/rt-audio-injection-colab
Try it out for yourself on free Colab TPUs!
g.co/magenta/rt-audio-injection-colab
Magenta RT now supports what we call *audio injection* - a novel method for real-time audio input
You can use audio injection to transform and add layers to prerecorded music!
You can use audio injection to transform and add layers to prerecorded music!
August 8, 2025 at 1:31 AM
Magenta RT now supports what we call *audio injection* - a novel method for real-time audio input
You can use audio injection to transform and add layers to prerecorded music!
You can use audio injection to transform and add layers to prerecorded music!
We’ve since been hard at work preparing new features. Check out our tech report on *Live Music Models*, including both Lyria RT (API) and Magenta RT (open)
📜 Paper: arxiv.org/abs/2508.046...
🔊Samples: g.co/magenta/live...
📜 Paper: arxiv.org/abs/2508.046...
🔊Samples: g.co/magenta/live...
Live Music Models
We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release Magenta RealTime, an ...
arxiv.org
August 8, 2025 at 1:31 AM
We’ve since been hard at work preparing new features. Check out our tech report on *Live Music Models*, including both Lyria RT (API) and Magenta RT (open)
📜 Paper: arxiv.org/abs/2508.046...
🔊Samples: g.co/magenta/live...
📜 Paper: arxiv.org/abs/2508.046...
🔊Samples: g.co/magenta/live...
ICYMI, in June we shared a preview of Magenta RealTime - the first open weights model capable of *live* music generation
🤗 Model: huggingface.co/google/magen...
⭐ Code: github.com/magenta/mage...
🤗 Model: huggingface.co/google/magen...
⭐ Code: github.com/magenta/mage...
google/magenta-realtime · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
August 8, 2025 at 1:31 AM
ICYMI, in June we shared a preview of Magenta RealTime - the first open weights model capable of *live* music generation
🤗 Model: huggingface.co/google/magen...
⭐ Code: github.com/magenta/mage...
🤗 Model: huggingface.co/google/magen...
⭐ Code: github.com/magenta/mage...
Looking forward to seeing all sorts of interesting ways you mangle and twist the outputs into unique creations 🙂.
July 9, 2025 at 6:30 PM
Looking forward to seeing all sorts of interesting ways you mangle and twist the outputs into unique creations 🙂.
We are excited to make these models more accessible, and more tightly integrated into musician’s workflows as it will open up more opportunities to customize and personalize how you use them to come up with truly personal expression.
July 9, 2025 at 6:30 PM
We are excited to make these models more accessible, and more tightly integrated into musician’s workflows as it will open up more opportunities to customize and personalize how you use them to come up with truly personal expression.
Built on the hard work of the Lyria Team, the VST plugin has been a labor of love of Ashutosh Desai. Thanks so much also to the support of artists Sidecar Tommy from Beats Antique, Toro y Moi in exploring the possibilities of DAW integration of our models.
July 9, 2025 at 6:30 PM
Built on the hard work of the Lyria Team, the VST plugin has been a labor of love of Ashutosh Desai. Thanks so much also to the support of artists Sidecar Tommy from Beats Antique, Toro y Moi in exploring the possibilities of DAW integration of our models.
The Infinite Crate is cross-platform, and currently supports Mac VST/AU, Windows VST, and a Mac Standalone application (no VST host required).
It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).
It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).
July 9, 2025 at 6:30 PM
The Infinite Crate is cross-platform, and currently supports Mac VST/AU, Windows VST, and a Mac Standalone application (no VST host required).
It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).
It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).
Be a Prompt DJ 🎧🎶: Play it like a live instrument. Use prompts and advanced controls to vary the instrumentation, texture, and genre. Incorporate with a DJ set to apply effects and mix with other loops.
July 9, 2025 at 6:30 PM
Be a Prompt DJ 🎧🎶: Play it like a live instrument. Use prompts and advanced controls to vary the instrumentation, texture, and genre. Incorporate with a DJ set to apply effects and mix with other loops.
Jam Along 🎸🥁: Create dynamic backing tracks that are different every time, varying and evolving as you play on top.
July 9, 2025 at 6:30 PM
Jam Along 🎸🥁: Create dynamic backing tracks that are different every time, varying and evolving as you play on top.
What can you do with it?
Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.
Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.
July 9, 2025 at 6:30 PM
What can you do with it?
Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.
Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.
Try it out and fork it here:
bit.ly/3ZOp7KS
Don't forget you can type in new prompts,
Curious to see what else you can make :).
bit.ly/3ZOp7KS
Don't forget you can type in new prompts,
Curious to see what else you can make :).
June 5, 2025 at 10:57 PM
Try it out and fork it here:
bit.ly/3ZOp7KS
Don't forget you can type in new prompts,
Curious to see what else you can make :).
bit.ly/3ZOp7KS
Don't forget you can type in new prompts,
Curious to see what else you can make :).
How are they being impacted by the current approach to deploying/integrating AI in our societal habitat? How could we be more sustainably approaching things?
April 23, 2025 at 5:31 PM
How are they being impacted by the current approach to deploying/integrating AI in our societal habitat? How could we be more sustainably approaching things?