Of course a bot had to chip in...
(no pun intended)

So Reinforcement Learning generally talks about agents, AlphaGo (Go playing AI) would be an example, etc.

(of course either way, this does not entail agency in a morally relevant sense, as the paper says)

Thanks for sharing! A minor point, but: "AI agents" being "specific to the AI safety and cybersecurity literature" seems overly narrow. In the context of AI I'd more generally understand an agent to be something that needs to autonomously take actions in an environment.

TLDR: Seth makes many valid points, but the case for biological naturalism isn't clear-cut IMO. In either case, I don't think his conclusion quite follows from the premises.
 
Ultimately, uncertainty still wins out for me.