Pawel Szczesny
@pawelszczesny.bsky.social
Now: evals, stability, psychology of Large Language Models @ Neurofusion Lab
Previously: R&D (academia & industry) in comp-bio, medtech, data science, VR psytech, nootropics.
Previously: R&D (academia & industry) in comp-bio, medtech, data science, VR psytech, nootropics.
There's non-linear relationship between temperature and instruction. When I ask OpenAI's 4o-mini about cardinal and intercardinal directions on a compass rose and start to swap words/phrases for synonyms, it turns out that some combinations give accuracy of 0%.
December 8, 2024 at 5:45 PM
There's non-linear relationship between temperature and instruction. When I ask OpenAI's 4o-mini about cardinal and intercardinal directions on a compass rose and start to swap words/phrases for synonyms, it turns out that some combinations give accuracy of 0%.
In one of my experiments I've tested what is distribution of scores assigned to a CV by a LLM when it's given a CV that is matching an offer and when it's not matching (instruction taken from a real ATS system). Variables: run (10 times) and synonyms in instruction.
Not bad, not great either.
Not bad, not great either.
December 8, 2024 at 5:39 PM
In one of my experiments I've tested what is distribution of scores assigned to a CV by a LLM when it's given a CV that is matching an offer and when it's not matching (instruction taken from a real ATS system). Variables: run (10 times) and synonyms in instruction.
Not bad, not great either.
Not bad, not great either.