With respect to the researchers' claims that the "the potential benefits of this research substantially outweigh its risks," it seems to mostly rely on "warning call" benefits.

I am personally not a fan of "we need to do harm to show that harm can be done" ethical justifications.

@narijohnson.bsky.social and Sanika have a FAccT paper about this (and also compiled cases of abandoned algorithms): arxiv.org/abs/2404.13802

This is one of the most basic insights from 20th century polisci.

The paradox of a democracy that presumes mass knowledge and participation, and manages to (at least barely) function in their absence, is one of our foundational puzzles.

This is intro-level shit. And Balaji is failing the test.

TL;DR MSFT being a leading GenAI company while also having researchers publishing work identifying the pitfalls of GenAI doesn’t imply hypocrisy in my opinion. Rather it reveals the existence of an open research environment where this type of critique can be empirically studied and published! 10/n

We also don't know if the authors did critique these larger systemic issues in an initial version and were asked to revise the paper to stay in scope! We only see the final version that is produced by both authors AND reviewers! 9/n

Authors can get dinged by reviewers for going outside their lane/presenting problems to which there is no technical solution or where the solution requires expertise beyond the scope of human-computer interaction. 8/n

BUT I am also someone who publishes in the same venue as this paper was published in and I know firsthand how difficult it is to introduce critiques of larger systems/organizational logics at CHI. 7/n

To be clear, I am generall critical of GenAI, and I do believe that companies like MSFT are complicit in shaping peoples’ attitudes towards AI systems through advertising campaigns and forceful product integration. 6/n

Fourth, the scope of this study is to evaluate the impacts of GenAI tools on critical thinking NOT to analyze the role of corporate marketing campaigns on knowledge workers’ overall behaviors and beliefs. The latter would be a great STS/critical paper! 5/n

It’s not the case that a handful of researchers, let alone a single intern, speak on behalf of a huge corporation like Microsoft lol. 4/n

Third, MSFT is not a monolith and in fact there are many researchers at MSR who put out work critiquing the tools, frameworks, and assumptions used throughout many of the GenAI products that MSFT the company makes 3/n