Dennis Aumiller
@daumiller.bsky.social
Getting paid to complain about LLM Evaluation at Cohere. #NLP #NLProc
https://dennis-aumiller.de
https://dennis-aumiller.de
Not super deep into the Docker/Run config, but do you have a strict requirement on having it be spun up for each call? Otherwise, you could also consider having a separate (static) eval server instance, akin to github.com/open-compass...
GitHub - open-compass/code-evaluator: A multi-language code evaluation tool.
A multi-language code evaluation tool. Contribute to open-compass/code-evaluator development by creating an account on GitHub.
github.com
January 30, 2025 at 11:35 PM
Not super deep into the Docker/Run config, but do you have a strict requirement on having it be spun up for each call? Otherwise, you could also consider having a separate (static) eval server instance, akin to github.com/open-compass...