Examples:
- GCG, PGD and other token-space optimization algorithms (optimize discrete F: token-sequence -> R)
- SAEs (why does a simple sparseness hack work so well?)

so any environment where there's a reliable "action done" event that you can catch programmatically works much much better

example trace from one of my experiments:
- claude clicks "expand"
- expand takes 2.5 seconds, so the next screenshot sent to claude (after ~2s) has no changes
- claude clicks again, but by that point expansion already happened, so the menu gets collapsed
- claude gets very confused and gives up

but also, some manual integration remains necessary. claude doesn't see a video, it sees screenshots with long gaps in between (the reference impl is like 2 seconds). how does claude know the application finished responding to the click, vs is still loading, vs the click didn't go through? it's hard

sorry i meant that desktop has those automations already so the marginal improvement is smaller than mobile.

but the killer use case will prob be smartphone app automation, it's something that's kinda hard to automate nowadays through other means. on desktop it's easier with things like applescript or windows desktop automate

imo it's still a little too dumb to be useful. we might need 6 months.