Galileo.ai
@rungalileo.bsky.social
The fastest way to ship reliable AI apps - Evaluation, Experimentation, and Observability Platform
Same with AI—if you throw an LLM at it and hope it'll figure itself out, you won't get the accuracy you want.”
To get the accuracy you’re looking for, you need to:
– Understand your data pipelines
– Test and evaluate continuously
– Treat infrastructure like your spellbook—essential for reliability
To get the accuracy you’re looking for, you need to:
– Understand your data pipelines
– Test and evaluate continuously
– Treat infrastructure like your spellbook—essential for reliability
June 11, 2025 at 9:52 PM
Same with AI—if you throw an LLM at it and hope it'll figure itself out, you won't get the accuracy you want.”
To get the accuracy you’re looking for, you need to:
– Understand your data pipelines
– Test and evaluate continuously
– Treat infrastructure like your spellbook—essential for reliability
To get the accuracy you’re looking for, you need to:
– Understand your data pipelines
– Test and evaluate continuously
– Treat infrastructure like your spellbook—essential for reliability
Greg Statton, at Cohesity, joins Conor Bronsdon on Chain of Thought, draws a sharp analogy between AI implementation and D&D:
💬 “AI is marketed as this magic bullet… but anyone who's played D&D knows—if you're a wizard trying to harness the power of the universe, you've got a lot of studying to do.
💬 “AI is marketed as this magic bullet… but anyone who's played D&D knows—if you're a wizard trying to harness the power of the universe, you've got a lot of studying to do.
June 11, 2025 at 9:52 PM
Greg Statton, at Cohesity, joins Conor Bronsdon on Chain of Thought, draws a sharp analogy between AI implementation and D&D:
💬 “AI is marketed as this magic bullet… but anyone who's played D&D knows—if you're a wizard trying to harness the power of the universe, you've got a lot of studying to do.
💬 “AI is marketed as this magic bullet… but anyone who's played D&D knows—if you're a wizard trying to harness the power of the universe, you've got a lot of studying to do.
Including Graph View, you now have three complementary ways to debug your agents:
→ Graph: Visualize decision paths and tool usage
→ Timeline: Spot performance bottlenecks instantly
→ Conversation: See the user experience end-to-end
→ Try these new views for yourself: app.galileo.ai/sign-up
→ Graph: Visualize decision paths and tool usage
→ Timeline: Spot performance bottlenecks instantly
→ Conversation: See the user experience end-to-end
→ Try these new views for yourself: app.galileo.ai/sign-up
June 11, 2025 at 6:01 PM
Including Graph View, you now have three complementary ways to debug your agents:
→ Graph: Visualize decision paths and tool usage
→ Timeline: Spot performance bottlenecks instantly
→ Conversation: See the user experience end-to-end
→ Try these new views for yourself: app.galileo.ai/sign-up
→ Graph: Visualize decision paths and tool usage
→ Timeline: Spot performance bottlenecks instantly
→ Conversation: See the user experience end-to-end
→ Try these new views for yourself: app.galileo.ai/sign-up
📊 Multi-tiered feedback loops in the wild: Learn how real-world reactions, iterative testing, and context-sensitive scoring reshape evaluation.
🎤 Comedy as a proving ground: See why humor is a great stress test for LLMs, and what it teaches us about creativity in AI.
🎤 Comedy as a proving ground: See why humor is a great stress test for LLMs, and what it teaches us about creativity in AI.
June 11, 2025 at 3:27 PM
📊 Multi-tiered feedback loops in the wild: Learn how real-world reactions, iterative testing, and context-sensitive scoring reshape evaluation.
🎤 Comedy as a proving ground: See why humor is a great stress test for LLMs, and what it teaches us about creativity in AI.
🎤 Comedy as a proving ground: See why humor is a great stress test for LLMs, and what it teaches us about creativity in AI.
You’ll hear what goes wrong (a lot), what we’re still learning about task-specific evaluation, & why evaluating funny is one of the hardest prompts in the game.
🌀 Chaos-tested LLM evaluation frameworks: Why standard metrics break down & what to use instead when the output is "lol" not "true/false."
🌀 Chaos-tested LLM evaluation frameworks: Why standard metrics break down & what to use instead when the output is "lol" not "true/false."
June 11, 2025 at 3:27 PM
You’ll hear what goes wrong (a lot), what we’re still learning about task-specific evaluation, & why evaluating funny is one of the hardest prompts in the game.
🌀 Chaos-tested LLM evaluation frameworks: Why standard metrics break down & what to use instead when the output is "lol" not "true/false."
🌀 Chaos-tested LLM evaluation frameworks: Why standard metrics break down & what to use instead when the output is "lol" not "true/false."
Check out the sessions!
🗓️ 6/10 Startup Forum Panel
www.databricks.com/dataaisummit...
🗓️ 6/11 Generating Laughter: Testing & Evaluating the Success of LLMs for Comedy
www.databricks.com/dataaisummit...
🗓️ 6/12 Taming Rogue AI Agents: Observability for Agentic Systems
www.databricks.com/dataaisummit...
🗓️ 6/10 Startup Forum Panel
www.databricks.com/dataaisummit...
🗓️ 6/11 Generating Laughter: Testing & Evaluating the Success of LLMs for Comedy
www.databricks.com/dataaisummit...
🗓️ 6/12 Taming Rogue AI Agents: Observability for Agentic Systems
www.databricks.com/dataaisummit...
Startup Forum | Databricks
Hear from VC leaders, startup founders and early stage customers building on Databricks around what they are seeing in the market and how they are scaling their early stage companies on Databricks. Th...
www.databricks.com
June 6, 2025 at 6:31 PM
Check out the sessions!
🗓️ 6/10 Startup Forum Panel
www.databricks.com/dataaisummit...
🗓️ 6/11 Generating Laughter: Testing & Evaluating the Success of LLMs for Comedy
www.databricks.com/dataaisummit...
🗓️ 6/12 Taming Rogue AI Agents: Observability for Agentic Systems
www.databricks.com/dataaisummit...
🗓️ 6/10 Startup Forum Panel
www.databricks.com/dataaisummit...
🗓️ 6/11 Generating Laughter: Testing & Evaluating the Success of LLMs for Comedy
www.databricks.com/dataaisummit...
🗓️ 6/12 Taming Rogue AI Agents: Observability for Agentic Systems
www.databricks.com/dataaisummit...
Enterprise AI isn't just about building responsibly - it's about proving it works safely at scale. When something goes wrong, you need to be able to explain why and how to fix it.
Ready to add that extra layer of AI evaluation to your enterprise systems? 🛡️
Ready to add that extra layer of AI evaluation to your enterprise systems? 🛡️
June 2, 2025 at 4:23 PM
Enterprise AI isn't just about building responsibly - it's about proving it works safely at scale. When something goes wrong, you need to be able to explain why and how to fix it.
Ready to add that extra layer of AI evaluation to your enterprise systems? 🛡️
Ready to add that extra layer of AI evaluation to your enterprise systems? 🛡️
➡️ Learn how to set up your MongoDB Atlas account and configure it with LangChain. Then we'll guide you through ingesting your data and utilizing the console to understand agent behavior and retriever tool performance.
📖 Read more: v2docs.galileo.ai/cookbooks/us...
📖 Read more: v2docs.galileo.ai/cookbooks/us...
MongoDB Atlas Integration for Retrieval-Augmented Generation (RAG) - Galileo
Guide to using MongoDB Atlas Vector Search with LangGraph agents logging to Galileo.
v2docs.galileo.ai
May 29, 2025 at 4:16 PM
➡️ Learn how to set up your MongoDB Atlas account and configure it with LangChain. Then we'll guide you through ingesting your data and utilizing the console to understand agent behavior and retriever tool performance.
📖 Read more: v2docs.galileo.ai/cookbooks/us...
📖 Read more: v2docs.galileo.ai/cookbooks/us...
Full blog post: www.galileo.ai/blog/ai-agen...
The AI Agent Evaluation Blueprint: Part 1
Playbook for building and shipping reliable Al Agents
www.galileo.ai
May 27, 2025 at 7:10 PM
Full blog post: www.galileo.ai/blog/ai-agen...