Aniket Ray
@aniketray.com
Agents @ Google, Reformed Startup Founder, Interested in Tech and Learning
Excellent read, always love a good rebuttal or replication. But don't necessarily agree with "..., we compared results with the same prompt. This is the only honest way to make any statements about structured vs unstructured performance.", we should likely be looking at optimal prompts for each
A new paper, "Let Me Speak Freely" has been spreading rumors that structured generation hurts LLM evaluation performance.
Well, we've taken a look and found serious issue in this paper, and shown, once again, that structured generation *improves* evaluation performance!
Well, we've taken a look and found serious issue in this paper, and shown, once again, that structured generation *improves* evaluation performance!
November 23, 2024 at 3:22 AM
Excellent read, always love a good rebuttal or replication. But don't necessarily agree with "..., we compared results with the same prompt. This is the only honest way to make any statements about structured vs unstructured performance.", we should likely be looking at optimal prompts for each
Anyone know how to bookmark here?
November 23, 2024 at 3:16 AM
Anyone know how to bookmark here?