Lightnews — Scholar-powered news

Reposted by NYU's Center for Social Media and Politics

Petter Törnberg @pettertornberg.com · 4d

LLMs are now widely used in social science as stand-ins for humans—assuming they can produce realistic, human-like text

But... can they? We don’t actually know.

In our new study, we develop a Computational Turing Test.

And our findings are striking:
LLMs may be far less human-like than we think.🧵

Computational Turing Test Reveals Systematic Differences Between Human and AI Language

Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior, based on the assumption that they can generate realistic, human-like text. Yet this assumption rem...

arxiv.org

14 130 320

Reposted by NYU's Center for Social Media and Politics

Tiago Ventura @tiagoventura.bsky.social · Oct 7

The paper is co-authored with Bernhard Von Clemm, @ericka.bric.digital
@jonathannagler.bsky.social @magdalenawojciesza.bsky.social

This is also one of the projects I started at @csmapnyu.org ---- thanks to the entire lab involved!

The paper can be found here: www.cambridge.org/core/journal...

Survey Professionalism: New Evidence from Web Browsing Data | Political Analysis | Cambridge Core

Survey Professionalism: New Evidence from Web Browsing Data

www.cambridge.org

1 9

Reposted by NYU's Center for Social Media and Politics

Tiago Ventura @tiagoventura.bsky.social · Oct 7

How common are “survey professionals” - people who take dozens of online surveys for pay - across online panels, and do they harm data quality?

Our paper, FirstView at @politicalanalysis.bsky.social, tackles this question using browsing data from three U.S. samples (Facebook, YouGov, and Lucid):

4 55 140

Reposted by NYU's Center for Social Media and Politics

Sarah Shugars @shugars.bsky.social · Aug 13

Fascinating work from @jatucker.bsky.social and @csmapnyu.org looking at the effect of labeling images as AI on people’s beliefs about the provenance + veracity of that image. They also explore the implications of AI images not being labeled after seeing labeled images #PaCSS2025 #polnet2025

2 8

Reposted by NYU's Center for Social Media and Politics

Sarah Shugars @shugars.bsky.social · Aug 13

Exciting methodological development from Ben Guinadeau and @csmapnyu.org using poisson factorization to do ideal point estimation of TikTok posts #pacss2025 #polnet2025

1 6

Reposted by NYU's Center for Social Media and Politics

Sarah Shugars @shugars.bsky.social · Aug 14

Very interesting work from @hwaight.bsky.social and @csmapnyu.org examining narrative similarity between news stories from global media sources. Methodological challenging because stories may communicate the same ideas/claims without using the same ngrams #pacss2025 #polnet2025

1 4 14

Reposted by NYU's Center for Social Media and Politics

Melina Much @melinamuch.bsky.social · Aug 14

My awesome co-author with our poster about the GenZ gender gap and podcast consumption at PolNet/PACSS! If you didn’t see us here catch us at APSA at the PolCom preconference and Saturday first thing in the morning 💃🏻 @csmapnyu.org

1 4

Reposted by NYU's Center for Social Media and Politics

Tiago Ventura @tiagoventura.bsky.social · Jul 17

🚨 Paper now as "just accepted" at @The_JOP. We ran the first WhatsApp deactivation experiment focused on multimedia content ahead of the 2022 election in Brazil. We find a reduction in users' recall of false rumors -- and, to a smaller degree, of true news. Null effects on attitudes. Full thread ⬇️

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

In the Global South, WhatsApp is more popular than X or Facebook.

New in @The_JOP, we ran a WhatsApp deactivation experiment during Brazil’s 2022 election to explore how the app facilitates the spread of misinformation and affects voters’ attitudes.

www.journals.uchicago.edu/doi/abs/10.1...

Abstract: In most advanced democracies, concerns about the spread of misinformation are typically associated with feed-based social media platforms like Twitter and Facebook. These platforms also account for the vast majority of research on the topic. However, in most of the world, particularly in Global South countries, misinformation often reaches citizens through social media messaging apps, particularly WhatsApp. To fill the resulting gap in the literature, we conducted a multimedia deactivation experiment to test the impact of reducing exposure to potential sources of misinformation on WhatsApp during the weeks leading up to the 2022 Presidential election in Brazil. We find that this intervention significantly reduced participants’ recall of false rumors circulating widely during the election. However, consistent with theories of mass media minimal effects, a short-term change in the information environment did not lead to significant changes in belief accuracy, political polarization, or well-being.

1 4 25

Reposted by NYU's Center for Social Media and Politics

Christopher Barrie @cbarrie.bsky.social · Jul 22

In addition to the original UK results, we have now ***replicated*** this (TWICE) in the US.

The main findings hold strong: information diets are a lot more diverse in attention than in engagement.

New version here: osf.io/preprints/os...

1 17 73

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Read a longer summary of the paper here: csmapnyu.org/research/aca...

Misinformation Beyond Traditional Feeds: Evidence from a WhatsApp Deactivation Experiment in Brazil - NYU’s Center for Social Media and Politics

csmapnyu.org

1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Congrats to the authors @tiagoventura.bsky.social, @rmajumdar.bsky.social, Jonathan Nagler, and @jatucker.bsky.social.

The paper, which is accepted for publication at @The_JOP, can be found here: www.journals.uchicago.edu/doi/epdf/10....

University of Chicago Press Journals: Cookie absent

www.journals.uchicago.edu

1 1 2

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Our study was a rare field experiment on misinformation in the Global South, adding to a growing call to broaden the geographic and platform scope of causally identified misinformation research.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Big takeaway: WhatsApp matters—but changing exposure does not mechanically change attitudes in the short run.

Political beliefs are hard to change and probably require long-term interventions.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

The paper also indicates:

➡️ Platforms like WhatsApp differ fundamentally from traditional feed-based platforms;

➡️ The academic community must spend time studying such platforms; and,

➡️ While difficult to study, it's crucial to explore how those in the Global Majority consume information

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

These findings have a key nuance: heavy WhatsApp users—those who frequently receive political content—did improve in spotting falsehoods, while others did not.

This suggests that information interventions may have unequal impacts across subgroups, depending on baseline exposure.

Marginal Treatment Effects of WhatsApp Multimedia Deactivation on Belief Accuracy conditional on using WhatsApp to receive and share information about politics

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Importantly, we also found no evidence that treated users substituted WhatsApp with other platforms.

Users did not migrate to Facebook, Instagram, or Telegram, but instead, watched a bit more TV.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

We are not the first, and unlikely to be the last, to find such mixed results.

Our findings echo recent Facebook deactivation studies & RCTs manipulating online informational spaces (e.g., re-shares, algorithmic feeds, landing pages) that show largely null effects on users’ attitudes.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

These findings are consistent with the “minimal effects” theory: the misinformation reduction did not translate to user belief accuracy and polarization changes.

Although users saw less false content, their attitudes stayed the same.

Treatment effects of WhatsApp Multimedia Deactivation on False Rumors Accuracy and True News Accuracy Judgments

Treatment Effects of WhatsApp Multimedia Deactivation on Polarization and Subjective Well-Being

1 1 4

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Overall, the recall of misinformation dropped sharply.

Participants were 40% less likely to remember false headlines, a significantly larger reduction than the decline in recall of true news.

Treatment effects of WhatsApp Multimedia Deactivation on self-reported exposure to false rumors and true news stories.

1 2

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

Our design recruited 773 WhatsApp users before the 2022 election. The treatment group (N≈400) turned off all auto-downloads (videos, images, audio, & docs) for 3 weeks, while the control group kept their usual settings. We also verified compliance weekly via storage screenshots.

Overview of the WhatsApp Multimedia Deactivation Experiment

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

We extend a well-known method for measuring social media’s causal effects in two directions:

1. Applying it to WhatsApp (not a feed-based platform).

2. Focusing on multimedia—the modal format of misinformation spread on WhatsApp.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

What makes WhatsApp unique?

Despite no news feed, millions of Brazilians use it as a key source of political information.

But it's also where they see the most misinformation, primarily via viral videos/images & group chats.

So we designed a deactivation experiment for this landscape.

1 1

NYU's Center for Social Media and Politics @csmapnyu.org · Jul 17

In the Global South, WhatsApp is more popular than X or Facebook.

New in @The_JOP, we ran a WhatsApp deactivation experiment during Brazil’s 2022 election to explore how the app facilitates the spread of misinformation and affects voters’ attitudes.

www.journals.uchicago.edu/doi/abs/10.1...

1 14 31

Reposted by NYU's Center for Social Media and Politics

Hannah Waight @hwaight.bsky.social · Jun 18

I am thrilled to share a new article in Sociological Methods & Research, “Quantifying Narrative Similarity Across Languages”. My co-first author Sol Messing and our collaborators developed a new approach to measuring “narrative similarity” between texts: journals.sagepub.com/doi/10.1177/...

Quantifying Narrative Similarity Across Languages - Hannah Waight, Solomon Messing, Anton Shirikov, Margaret E. Roberts, Jonathan Nagler, Jason Greenfield, Megan A. Brown, Kevin Aslett, Joshua A. Tuck...

How can one understand the spread of ideas across text data? This is a key measurement problem in sociological inquiry, from the study of how interest groups sh...

journals.sagepub.com

3 27 57

Reposted by NYU's Center for Social Media and Politics

Andreu Casas @andreucasas.bsky.social · Apr 23

🚨New Publication @thejop.bsky.social We study who sets the issue agenda of state policymakers in the US. We find a mixed picture: they respond to shifts in attention by members of Congress, but also to the constituents in their states, particularly their own party supporters.

NYU's Center for Social Media and Politics @csmapnyu.org · Apr 21

Amid growing DC gridlock, state legislatures play a vital role in shaping policy. But how do state lawmakers decide which issues deserve their attention? Our new @thejop.bsky.social paper is the first large-scale multi-state analysis exploring this question

www.journals.uchicago.edu/doi/10.1086/...

Title: Bottom Up? Top Down? Determinants ofIssue-Attention in State Politics

Abstract: Who shapes the issue-attention cycle of state legislators? Although state governments make critical policy decisions, data and methodological constraints have limited re-searchers’ ability to study state-level agenda setting. For this paper, we collect more than 122 million Twitter messages sent by state and national actors in 2018 and 2021.We then employ supervised machine learning and time series techniques to study how the issue-attention of state lawmakers evolves vis-`a-vis various local- and national-level actors. Our findings suggest that state legislators operate at the confluence of national and local influences. In line with arguments highlighting the nationalization of state politics, we find that state legislators are consistently responsive to policy debates among members of Congress. However, despite growing nationalization concerns, we also find strong evidence of issue responsiveness by legislators to members of the public in their states and moderate responsiveness to regional media sources.

6 3 8