I look forward to the post I assume this is leading to for further enlightenment - I've been long puzzled by how seemingly simple/small the code around the giant pile of weights seems to be in practice, these sorts of insights are awesome.

huh, this (cache value, being a pure functional mapping from the input prompt) is a TIL moment for me...

I naively assumed that the point of prompt caching was about restoring internal state of the model...!

In hindsight that assumption seems obviously dumb, given the sizes involved!

Thanks :)

But IMO just as easy to put the server in a secured room with badge access logs, etc, and/or a locked rack in the corner of the office vs under a desk.

I don't think a compromised dev machine is comparable - they should *never* have secrets directly exposed them (vs CI/CD which requires them)...

Yes, SOC2 in my experience is mostly validating that you have a set of policies and controls in place, that you assert are suitable for your business (vs a very low-bar baseline) and that you actually follow them.

So if you want to declare this not a risk, your auditor will probably accept it.

I'd be more worried about the security/supply chain risks:

Assumption: You sell a product to/maintain OSS used by someone important that attacker X wants to compromise.

Threat model: X breaks into your office, compromises your under-desk CI server with subtle malware that backdoors your builds.

Under-desk (vs on-prem server room) also raises physical security questions (e.g. evil maid/cleaner attack) that I would find harder to justify SOC2/ISO controls against.

A CI server is riskier than a dev desktop - it deploys directly to prod, while desktop actions are gated through a review step.

I'd look at it less from a reliability perspective and more from maintenance and security.

Under-desk might be fine if it's well-managed (updated, monitored, etc) but "spare box" has connotations that point away from that...

Is the under-desk runner in your MDM/inventory and regularly updated?

watching with interesting, and intruiged by the idea, but timezones are challenging...

If/when you have an iteration of this that works for UTC+12/UTC+13 (NZ) I would be interested.

added to my queue, but do you know why the transistor share page doesn't link to Spotify?

I had to spend an extra minute manually searching for it in Spotify...

I'm guessing #835 having just done all 3...

I got it 3rd, but purely by guessing/segmenting the 8 remaining words into which 4 seemed most likely to match some weird american grouping - a tactic I have to use frequently!

is "today" for you #834, #835 or #836 ?

Timezones make this hard :)

Yes!

I think this is the next natural opportunity for oncall-optimizer.com to expand into....

noun verb

how/where do you review Claude Code's output without an editor?

1 minute of confusion, but otherwise easy:

1) Google "nz current covid case data"
2) Click first result (https://www.tewhatuora.govt.nz/for-health-professionals/data-and-statistics/covid-19-data/covid-19-case-demographics)
3) Navigate breadcrumbs to "Current Cases" (parent page + first link there)

This was super encouraging to consider on day ~700 of my current journey, thanks!

Do you have a longer article on this topic too? I'm particularly interested in whether 1000 is just a nice round number that feels right from your experience, or whether there's something deeper...

terminal testing also covers both me and claude code, probably more claude code than me these days :)

1) manually from the terminal during dev (specific tests, relevant to code under dev)
2) pre-push (via a git hook and the pre-commit framework, full repo suite)
3) as a branch compliance/merge requirement (via GitHub Actions) - the most important one, in a known good environment, etc).