I'd recommend this course for anyone working on operational systems where LLM output may play a load-bearing role, even if you're not of the engineering persuasion, but are up for getting your hands dirty with some code.

maven.com/parlance-lab...

The course validated my intuitions that these foundations apply just as much to generative AI applications and has given me a road-tested pragmatic framework for applying them to the weird world of LLMs and AI engineering.

Their approach thoroughly resonates with my experience as a data scientist: look at your data, perform thorough error analysis, develop metrics that matter, and iterate via experimentation and user-feedback.

I recently completed @hamel.bsky.social and @sh-reya.bsky.social's AI Evals For Engineers & PMs, which presented practical frameworks for identifying appropriate dimensions of quality, creating automatic validators to measure at scale, and effective feedback loops that enable continual improvement