Luke Marsden
@lmarsden.bsky.social
Hacker & entrepreneur. Founder helix.ml, private GenAI stack, getting business value out of local open source LLMs
Thanks Hannah! 💖
Always a good sign when the speakers are mobbed after their talk! Well done @lmarsden.bsky.social and Priya - smashed it! #kubecon #cloudnativecon
April 3, 2025 at 7:26 AM
Thanks Hannah! 💖
Good morning #KubeCon! Come find me if you want to talk running LLM systems like vision RAG in production on Kubernetes today. Email luke@helix.ml or reply here!
April 3, 2025 at 7:25 AM
Good morning #KubeCon! Come find me if you want to talk running LLM systems like vision RAG in production on Kubernetes today. Email luke@helix.ml or reply here!
Catch my talk with Priya Samuel on running LLMs in production on k8s today at #KubeCon at S10A at 3:15pm ✨
Trying (but failing) to be as colourful as @hannahfoxwell.net today!
April 2, 2025 at 7:56 AM
Catch my talk with Priya Samuel on running LLMs in production on k8s today at #KubeCon at S10A at 3:15pm ✨
Trying (but failing) to be as colourful as @hannahfoxwell.net today!
April 2, 2025 at 7:54 AM
Trying (but failing) to be as colourful as @hannahfoxwell.net today!
Reposted by Luke Marsden
Oh hello there BlueSky! We've arrived here just in time to share all the highlights of #KubeCon #CloudNativeCon in sunny London!
Don't miss @hannahfoxwell.net and @lmarsden.bsky.social at 15:20 today talking about Platform Engineering and Developer Experience For Your On Prem LLM
Don't miss @hannahfoxwell.net and @lmarsden.bsky.social at 15:20 today talking about Platform Engineering and Developer Experience For Your On Prem LLM
April 1, 2025 at 12:41 PM
Oh hello there BlueSky! We've arrived here just in time to share all the highlights of #KubeCon #CloudNativeCon in sunny London!
Don't miss @hannahfoxwell.net and @lmarsden.bsky.social at 15:20 today talking about Platform Engineering and Developer Experience For Your On Prem LLM
Don't miss @hannahfoxwell.net and @lmarsden.bsky.social at 15:20 today talking about Platform Engineering and Developer Experience For Your On Prem LLM
Reposted by Luke Marsden
I looked to define a term that’s being thrown around — #AI-native developer — with the help of @guypo.com @lmarsden.bsky.social and Thoughtworks’ Mike Mason so you can prepare your software teams for the near future.
thenewstack.io/what-is-an-a...
What’s your definition? Only on @thenewstack.io
thenewstack.io/what-is-an-a...
What’s your definition? Only on @thenewstack.io
What Is an AI Native Developer?
We explain what an AI native developer is, how the role will evolve, and why spec-driven AI development is driving this trend.
thenewstack.io
February 20, 2025 at 5:47 PM
I looked to define a term that’s being thrown around — #AI-native developer — with the help of @guypo.com @lmarsden.bsky.social and Thoughtworks’ Mike Mason so you can prepare your software teams for the near future.
thenewstack.io/what-is-an-a...
What’s your definition? Only on @thenewstack.io
thenewstack.io/what-is-an-a...
What’s your definition? Only on @thenewstack.io
Deepseek are clever fuckers. I wrote this about how Deepseek is pushing decision makers in large financial institutions to seriously consider running their own models instead of calling out to Microsoft, Amazon & Google
Link below 👇
Link below 👇
January 30, 2025 at 10:01 AM
Deepseek are clever fuckers. I wrote this about how Deepseek is pushing decision makers in large financial institutions to seriously consider running their own models instead of calling out to Microsoft, Amazon & Google
Link below 👇
Link below 👇
Reposted by Luke Marsden
Speaker @lmarsden.bsky.social of MLOps Consulting will be in our SOOCon25 AI Openness track! 🎤 Join to hear about Production-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performance.💡https://buff.ly/3HjeQxq
#opensource #opensourceai #ai #stateofopencon #soocon25 #opensourcelondon #openuk
#opensource #opensourceai #ai #stateofopencon #soocon25 #opensourcelondon #openuk
January 15, 2025 at 5:00 PM
Speaker @lmarsden.bsky.social of MLOps Consulting will be in our SOOCon25 AI Openness track! 🎤 Join to hear about Production-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performance.💡https://buff.ly/3HjeQxq
#opensource #opensourceai #ai #stateofopencon #soocon25 #opensourcelondon #openuk
#opensource #opensourceai #ai #stateofopencon #soocon25 #opensourcelondon #openuk
Reposted by Luke Marsden
I enjoyed Stories of Your Life and Others so much that I'm moving straight to Chiang's other collection: Exhalation.
January 12, 2025 at 7:59 AM
I enjoyed Stories of Your Life and Others so much that I'm moving straight to Chiang's other collection: Exhalation.
MS Teams in the UK this morning is the most grim experience imaginable, tons of jitter and bandwidth seems fucked - like trying to do a video call over a highly contended LTE connection - my network connection is fine (per mtr) - is there an outage in some MS data center somewhere?
January 8, 2025 at 11:34 AM
MS Teams in the UK this morning is the most grim experience imaginable, tons of jitter and bandwidth seems fucked - like trying to do a video call over a highly contended LTE connection - my network connection is fine (per mtr) - is there an outage in some MS data center somewhere?
We shipped a lot over Christmas and I came here to release it:
github.com/helixml/heli...
- You can now drag'n'drop files directly into knowledge for Helix apps (rather than having to go via the filestore).
- Initial support for MCP (more on this coming soon!)
github.com/helixml/heli...
- You can now drag'n'drop files directly into knowledge for Helix apps (rather than having to go via the filestore).
- Initial support for MCP (more on this coming soon!)
Release 1.4.12 - apps drag'n'drop filestore, initial MCP support · helixml/helix
What's Changed
make frontend work better with filestore knowledge by @nessie993 in #668
Model Context Protocol (MCP) support by @nessie993 in #680
security/refactoring/quality changes:
Revive vo...
github.com
January 6, 2025 at 4:19 PM
We shipped a lot over Christmas and I came here to release it:
github.com/helixml/heli...
- You can now drag'n'drop files directly into knowledge for Helix apps (rather than having to go via the filestore).
- Initial support for MCP (more on this coming soon!)
github.com/helixml/heli...
- You can now drag'n'drop files directly into knowledge for Helix apps (rather than having to go via the filestore).
- Initial support for MCP (more on this coming soon!)
Cancelled my X subscription over all the stupid political interference
January 3, 2025 at 8:50 PM
Cancelled my X subscription over all the stupid political interference
Thank you to @dciangot.com for doing the heavy lifting getting HelixML GPU runners running on Slurm HPC infra to take advantage of hundreds of thousands of GPUs running on Slurm infrastructure and transform them into multi-tenant GenAI systems!
Read all about it here: blog.helix.ml/p/running-ge...
Read all about it here: blog.helix.ml/p/running-ge...
Running GenAI on Supercomputers: Bridging HPC and Modern AI Infrastructure
Transform Your Supercomputer into a Private OpenAI: A Look at How Modern HPC Infrastructure Can Power Enterprise AI
blog.helix.ml
December 20, 2024 at 11:00 AM
Thank you to @dciangot.com for doing the heavy lifting getting HelixML GPU runners running on Slurm HPC infra to take advantage of hundreds of thousands of GPUs running on Slurm infrastructure and transform them into multi-tenant GenAI systems!
Read all about it here: blog.helix.ml/p/running-ge...
Read all about it here: blog.helix.ml/p/running-ge...
This is so awesome! Imagine being able to combine the worlds of cloud-native scalable web services for LLMOps with the raw power of Slurm-powered supercomputers with some of the biggest compute power, networking and GPUs! Check out the writeup here: blog.helix.ml/p/running-ge...
December 20, 2024 at 10:58 AM
This is so awesome! Imagine being able to combine the worlds of cloud-native scalable web services for LLMOps with the raw power of Slurm-powered supercomputers with some of the biggest compute power, networking and GPUs! Check out the writeup here: blog.helix.ml/p/running-ge...
So @cybernetist.com got gptscript working with llama3.3:70b! Check out the detailed writeup here: blog.helix.ml/p/gptscript-...
GPTScript Helix Apps: For Fun and Profit
Making GPTScript Shine with Open Source LLMs: How Llama 3's 70B Model Finally Makes Natural Language Programming Reliable
blog.helix.ml
December 19, 2024 at 5:25 PM
So @cybernetist.com got gptscript working with llama3.3:70b! Check out the detailed writeup here: blog.helix.ml/p/gptscript-...
Got helix.ml running GenAI on a supercomputer..
Private GenAI Platform – HelixML
Build, test and deploy GenAI apps. Connect knowledge and business applications via APIs. Deploy in your cloud or use ours
helix.ml
December 13, 2024 at 7:04 PM
Got helix.ml running GenAI on a supercomputer..
Fucking dope helix.ml release from @cybernetist.com if you like function calling in open source LLMs and @GPTScript_ai github.com/helixml/heli...
(Excuse my French)
(Excuse my French)
Release 1.4.9 - support function calling in helix runners, add gptscript runner · helixml/helix
What's Changed
Add gptscript_runner service to helix stack by @milosgajdos in #635
Bump ollama and update LLM inference with Tool calling by @milosgajdos in #637
Full Changelog: 1.4.8...1.4.9
github.com
December 11, 2024 at 7:09 PM
Fucking dope helix.ml release from @cybernetist.com if you like function calling in open source LLMs and @GPTScript_ai github.com/helixml/heli...
(Excuse my French)
(Excuse my French)
Test Driven Development (TDD) for your LLMs? Yes please, more of that please!
Back to basics - write a test, see the test fail, improve the prompt, see it pass, check it in - just like you would with any other code 😄
Back to basics - write a test, see the test fail, improve the prompt, see it pass, check it in - just like you would with any other code 😄
December 4, 2024 at 2:41 PM
Test Driven Development (TDD) for your LLMs? Yes please, more of that please!
Back to basics - write a test, see the test fail, improve the prompt, see it pass, check it in - just like you would with any other code 😄
Back to basics - write a test, see the test fail, improve the prompt, see it pass, check it in - just like you would with any other code 😄
Reposted by Luke Marsden
Luke, this post, including the video, was very helpful. I loved the conference talk. I took my graduate students through a somewhat similar exercise last week for their last lab of the semester. Well done. Thank you.
November 28, 2024 at 10:58 AM
Luke, this post, including the video, was very helpful. I loved the conference talk. I took my graduate students through a somewhat similar exercise last week for their last lab of the semester. Well done. Thank you.
Reposted by Luke Marsden
This is a wonderful post by @lmarsden.bsky.social Great presentation and video as well.
ICYMI, here's my 2-part blog series on how we can all be AI engineers with open source models – and how to apply testing best practices to LLM apps
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
We can all be AI engineers – and we can do it with open source models
The barriers to AI engineering are crumbling fast
blog.helix.ml
November 28, 2024 at 11:00 AM
This is a wonderful post by @lmarsden.bsky.social Great presentation and video as well.
ICYMI, here's my 2-part blog series on how we can all be AI engineers with open source models – and how to apply testing best practices to LLM apps
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
We can all be AI engineers – and we can do it with open source models
The barriers to AI engineering are crumbling fast
blog.helix.ml
November 28, 2024 at 9:49 AM
ICYMI, here's my 2-part blog series on how we can all be AI engineers with open source models – and how to apply testing best practices to LLM apps
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
1. blog.helix.ml/p/we-can-all...
2. blog.helix.ml/p/from-click...
@llm360.bsky.social please consider adding me to the open source LLMs starter pack ☺️
November 23, 2024 at 9:34 PM
@llm360.bsky.social please consider adding me to the open source LLMs starter pack ☺️
So how can we _actually_ do AI on Kubernetes? Here's my humble proposal: LLM apps and tests as data (CRDs). Deeply integrated into the existing CI/CD pipeline e.g GitLab and Flux. Self hosted LLMs on GPUs if you want to. Runnable code: github.com/helixml/gena...
GitHub - helixml/genai-cicd-ref: Cloud-Native GenAI Reference Architecture
Cloud-Native GenAI Reference Architecture. Contribute to helixml/genai-cicd-ref development by creating an account on GitHub.
github.com
November 23, 2024 at 11:13 AM
So how can we _actually_ do AI on Kubernetes? Here's my humble proposal: LLM apps and tests as data (CRDs). Deeply integrated into the existing CI/CD pipeline e.g GitLab and Flux. Self hosted LLMs on GPUs if you want to. Runnable code: github.com/helixml/gena...