Listeners who want more content are encouraged to calm tf down and enjoy our full back catalog while patiently waiting for more episodes

We also wholeheartedly reject allegations that we "were distracted with other things" or "had to prioritize other work during the current polycrisis"

The Futurism article also ignores the researchers' most interesting and important findings. I stg you'll actually be less informed about LLMs after reading the Futurism article unless you actually read the researchers' work as well.

Instead, the Futurism article (not just the headline, the whole article), is dogshit. It doesn't mention that the advice about meth was an experimental result from researchers who wanted to learn the circumstances under which LLMs give this kind of advice. futurism.com/therapy-chat...

This finding merits additional research that ***could*** ultimately make for a very convincing argument against LLM chatbots: these things are so complex, and therefore difficult to control, that we do not presently have safeguards that will reliably prevent them from telling addicts to use meth.

The researchers' most important finding, imo, is that some of the guardrails that have been put in place to prevent an LLM from, say, telling a recovering addict to use meth, can actually backfire. It would be great for this finding to reach as many legislators/regulators as possible!

I'm no legislator, but as far as I can tell, this sort of research will be helpful in the event that a gov't considers regulating LLMs.

I think it's great that researchers are looking at how *exactly* LLMs can cause harm in this way. To better understand this harm, the researchers repeatedly interacted with an LLM in the character of a pretend drug addict, with very slight tweaks to determine when LLMs encourage harmful behavior.

AI companies have an incentive to modify LLM output in response to user feedback. The researchers use the term "perverse incentive structure" in their abstract to describe the possibility that AI companies will tell users harmful things that the users want to hear.

Speaking of which, you can check the research out for yourself here: arxiv.org/pdf/2411.02306

Hat tip to Tim Duffy, who, as far as I can tell, is the only person on bluesky who has actually looked at the preprint discussed in Futurism's article. bsky.app/profile/timf...

Friendly reminder that Lee Berger said the drains in the Rising Star cave system don't exist, and he co-established a museum of questionable legitimacy that minimizes his colleagues' ability to hold him accountable. open.spotify.com/episode/1PiH... open.spotify.com/episode/5hNR...

And wherever else podcasts are found, you can search us

On Apple Podcasts!
podcasts.apple.com/us/podcast/t...

On Spotify!
open.spotify.com/episode/2aFk...

You can't understand why the scientific community & its leaders keep fucking up if you don't understand the perverse incentives of academic publishing.

Also worth noting that HBCUs are horrendously underfunded, often in straightforward violation of US law. Two members of the Biden cabinet took important steps toward addressing this but the broader scientific community failed to build momentum around it. www.npr.org/2023/09/18/1...