@zashktorab.bsky.social
So excited that this is open-sourced now! Our team has worked to make building custom criteria for LLM-as-a-judge easy. Check out the link, repo, and research below!
📣 Today we open-sourced EvalAssist, a web-based tool that makes it super easy to develop criteria for llm judges. You can run this now locally and then scale up with notebooks using Unitxt. Check out the AI Alliance article to get the scoop:
thealliance.ai/blog/llm-as-...
thealliance.ai/blog/llm-as-...
LLM-as-a-Judge Without the Headaches: EvalAssist Brings Structure and Simplicity to the Chaos of LLM Output Review | AI Alliance
Evaluating AI model outputs at scale is a major challenge for teams using LLMs, especially when assessing nuanced qualities like politeness, fairness, and tone that traditional benchmarks miss. IBM Re...
thealliance.ai
June 16, 2025 at 11:17 PM
So excited that this is open-sourced now! Our team has worked to make building custom criteria for LLM-as-a-judge easy. Check out the link, repo, and research below!