It's very life-like

"minimising ... sycophancy"

My kingdom for a solution to this.

It was using for nix & cabal rather than for writing code. This kept the context small and I could feed it flake files and errors/short problem statements and ask for solutions. The solutions could be tested easily, and unambiguously worked or didn't. So if it made stuff up I could tell quickly.

"I do not care for Martin-Löf.
Would you please just bugger off"

I've seen tests like this where at first glance most of the tests look fine, but then you realise they're just doing work and not actually testing anything.