Ivan Leo
@ivanleomk.bsky.social
Applied AI stuff from time to time, I write at ivanleo.com
A common problem I see is people expecting models to solve all their problems in a single try.
Instead treat it as an iterative process and work slowly towards building a portfolio of good outputs and reviewing them consistently
ivanleo.com/blog/youre-p...
Instead treat it as an iterative process and work slowly towards building a portfolio of good outputs and reviewing them consistently
ivanleo.com/blog/youre-p...
You're probably using LLMs wrongly - Ivan's Blog
Get 10x the results with LLMs in 3 simple steps
ivanleo.com
November 27, 2024 at 7:23 AM
A common problem I see is people expecting models to solve all their problems in a single try.
Instead treat it as an iterative process and work slowly towards building a portfolio of good outputs and reviewing them consistently
ivanleo.com/blog/youre-p...
Instead treat it as an iterative process and work slowly towards building a portfolio of good outputs and reviewing them consistently
ivanleo.com/blog/youre-p...
My favourite evaluation is an assert statement and it works surprisingly well. Stupid evals helps us turn vibes into objective metrics - one step at a time.
See my favourite stupid evals at ivanleo.com/blog/write-s...
See my favourite stupid evals at ivanleo.com/blog/write-s...
Write Stupid Evals - Ivan's Blog
Keep it simple and worry about the rest later
ivanleo.com
November 26, 2024 at 4:00 AM
My favourite evaluation is an assert statement and it works surprisingly well. Stupid evals helps us turn vibes into objective metrics - one step at a time.
See my favourite stupid evals at ivanleo.com/blog/write-s...
See my favourite stupid evals at ivanleo.com/blog/write-s...
Reposted by Ivan Leo
Evals are "too damn expensive" until you:
• can't migrate underlying models safely
• can't add new features with confidence
• can't ship without HITL evals, which takes >100x longer
• product development and iteration grinds to a halt
• lose customer trust due to poor user experience
• can't migrate underlying models safely
• can't add new features with confidence
• can't ship without HITL evals, which takes >100x longer
• product development and iteration grinds to a halt
• lose customer trust due to poor user experience
November 23, 2024 at 4:57 AM
Evals are "too damn expensive" until you:
• can't migrate underlying models safely
• can't add new features with confidence
• can't ship without HITL evals, which takes >100x longer
• product development and iteration grinds to a halt
• lose customer trust due to poor user experience
• can't migrate underlying models safely
• can't add new features with confidence
• can't ship without HITL evals, which takes >100x longer
• product development and iteration grinds to a halt
• lose customer trust due to poor user experience
Spent some time writing about my takeaways from running loads of experiments as a new data scientists working with LLMs
www.ivanleo.com/blog/youre-p...
www.ivanleo.com/blog/youre-p...
You're probably not doing experiments right - Ivan's Blog
How to get the results that you want from your LLM experiments
www.ivanleo.com
November 20, 2024 at 3:04 PM
Spent some time writing about my takeaways from running loads of experiments as a new data scientists working with LLMs
www.ivanleo.com/blog/youre-p...
www.ivanleo.com/blog/youre-p...