MLflow
@mlflow.org
An open source machine learning platform for managing the complete ML lifecycle
With that metadata—and MLflow’s MCP features released recently—the judge can make tool calls to MLflow to search spans and query different aspects of the trace.
Office Hours: Wed, Oct 22: luma.com/officehours1...
#mlflow #agenticjudges #llm #genai
Office Hours: Wed, Oct 22: luma.com/officehours1...
#mlflow #agenticjudges #llm #genai
MLflow Office Hours | October 22 · Zoom · Luma
luma.com
October 15, 2025 at 2:15 PM
With that metadata—and MLflow’s MCP features released recently—the judge can make tool calls to MLflow to search spans and query different aspects of the trace.
Office Hours: Wed, Oct 22: luma.com/officehours1...
#mlflow #agenticjudges #llm #genai
Office Hours: Wed, Oct 22: luma.com/officehours1...
#mlflow #agenticjudges #llm #genai
In this mode, the judge gets the MLflow trace info object: input to the call, output, and basically the root span ID for that trace.
October 15, 2025 at 2:15 PM
In this mode, the judge gets the MLflow trace info object: input to the call, output, and basically the root span ID for that trace.
The real unlock → MLflow’s tracing integration. Every tool call + reasoning step gets captured and replayable. When an agent fails, you can see why—not guess. Critical for debugging multi-step chains + production bottlenecks.
🔗 Learn more: mlflow.org/docs/latest/...
#LLMOps #AI #MLflow #oss
🔗 Learn more: mlflow.org/docs/latest/...
#LLMOps #AI #MLflow #oss
Evaluating Agents | MLflow
AI Agents are an emerging pattern of GenAI applications that can use tools, make decisions, and execute multi-step workflows. However, evaluating the performance of those complex agents is challenging...
mlflow.org
September 5, 2025 at 3:06 PM
The real unlock → MLflow’s tracing integration. Every tool call + reasoning step gets captured and replayable. When an agent fails, you can see why—not guess. Critical for debugging multi-step chains + production bottlenecks.
🔗 Learn more: mlflow.org/docs/latest/...
#LLMOps #AI #MLflow #oss
🔗 Learn more: mlflow.org/docs/latest/...
#LLMOps #AI #MLflow #oss
MLflow lets you create custom scorers for agent behavior: did it use the right tool, in the right order, with proper reasoning? Datasets can encode patterns + decisions, not just input–output. You’re testing how the agent thinks—not just what it outputs.
#AgentEvaluation #MLflow #opensource
#AgentEvaluation #MLflow #opensource
September 5, 2025 at 3:05 PM
MLflow lets you create custom scorers for agent behavior: did it use the right tool, in the right order, with proper reasoning? Datasets can encode patterns + decisions, not just input–output. You’re testing how the agent thinks—not just what it outputs.
#AgentEvaluation #MLflow #opensource
#AgentEvaluation #MLflow #opensource