Takeaway: We can't just play "whack-a-mole" with content. To build healthier online spaces, we have to start being architects.

Thanks to @knightfoundation.org for financial support!

Our full paper is here: www.nature.com/articles/s41...

The real culprit? Platform Architecture.

The fundamental design of the platform—its rules, its affordances, its very blueprint—allowed these decentralized communities to form and thrive. The system itself was perfectly built to resist the very kind of top-down control that was attempted.

So why did some people think it was working?

If you only look at the immediate neighborhood of banned accounts, it seems like a success. But when we zoomed out, we saw the problem was just growing and shifting across the wider network.

We found that the top-down content moderation policies actually triggered "reactance"—a powerful backlash that strengthened the targeted communities.

Instead of silencing them, the bans became a threat that reinforced their identity and made them more resilient.

Thanks for the note! Your work absolutely inspired ours and the “one size fits all” threshold refers to another paper by another set of authors.

Proud to work with an amazing team: @yunkangyang.bsky.social , Ramesh Paudel, Jordan McShan, @matthindman.bsky.social , and Howie Huang.

And grateful to the Knight Foundation for supporting work on transparent methods for information integrity.

Full paper: doi.org/10.1038/s415...

This dual-detection strategy is key to spotting the more subtle stuff. This paper builds on our broader interest in platform dynamics and misinformation. But the method is applicable to any kind of content diffusion—hashtags, images, even AI-generated text.

📊 About 23% of pages showed signs of coordination—some of it expected (e.g., news syndication), but a lot of it less transparent.

The method helped us identify both overt and covert networks. Some shared links every few seconds. Others did it slowly but repeatedly, evading the usual red flags.

Using mixture models, we derive empirically grounded thresholds that tell us when the pattern is too consistent to be chance.

No guesswork. No black box.

We applied this to over 11 million Facebook posts from 16K high-engagement pages.

Most detection methods use arbitrary cutoffs (like “shared within 10 seconds”), which don’t hold up across platforms or contexts.

We do better.

Our method looks at two things:
1️⃣ How fast two pages share the same link
2️⃣ How often they do it over time

Our approach is platform-agnostic, scalable, and designed to support transparency and accountability. Why does this matter?

Coordinated sharing—whether by media groups, advocacy networks, or covert actors—can shape what people see online.

This project -- led by
@lorien.bsky.social , in collaboration with
Artin Yousefi, Christina Wysota, and Tien-Chin Wu -- was supported by a TRAILS seed grant.

Better AI can help people make more informed, healthier decisions! 🚭🤖

7️⃣ So, what’s the takeaway?

🔹 Chatbots can be helpful, but they need better training.
🔹 WHO’s Sarah outperformed the others.
🔹 AI can easily be prompted to give weird advice. 😵‍💫

6️⃣ Surprisingly, the chatbots resisted adversarial attacks!

Even with tricky prompts, they stayed on topic & avoided harmful advice.

5️⃣ Queries on quitting "cold turkey" or with vapes, gummies, or hypnosis? The chatbots STRUGGLED. 🥴

4️⃣ Bad news: Some responses contained unsubstantiated claims (22%) 😱

For example, an AI recommended using a necklace to quit smoking. 😭

3️⃣ Good news: Chatbots mostly recommended professional help (80%) and used clear language (97%).

2️⃣ Across chatbots, responses followed guidelines only 57% of the time.

Sarah led with 72% adherence, while BeFreeGPT & BasicGPT lagged at ~50%. 😬

1️⃣ First, we tested three AI chatbots: WHO’s Sarah, BeFreeGPT, and BasicGPT.

Do they give reliable quitting advice? Let’s find out! 🔎