klaudia jaźwińska
@klaudia.bsky.social
researching journalism x tech for @towcenter.bsky.social, writing in @columjournreview.bsky.social
📍 on the amtrak between philly and nyc
klaudiajazwinska.com
📍 on the amtrak between philly and nyc
klaudiajazwinska.com
damn there's a fake, likely AI-generated quote attributed to me in an article about AI. I've never used the phrase "Faustian bargain" in my life
September 26, 2025 at 2:20 PM
damn there's a fake, likely AI-generated quote attributed to me in an article about AI. I've never used the phrase "Faustian bargain" in my life
When @aisvarya17.bsky.social and I asked seven different AI models to identify the location, date and source of photos taken by photojournalists, they collectively only answered 14 out of 280 queries completely correctly.
August 26, 2025 at 10:10 PM
When @aisvarya17.bsky.social and I asked seven different AI models to identify the location, date and source of photos taken by photojournalists, they collectively only answered 14 out of 280 queries completely correctly.
As in our past research into AI search tools, we observed instances where the models were confidently wrong and other instances where they hedged, despite providing correct answers. www.cjr.org/tow_center/w...
August 26, 2025 at 8:46 PM
As in our past research into AI search tools, we observed instances where the models were confidently wrong and other instances where they hedged, despite providing correct answers. www.cjr.org/tow_center/w...
However, they also struggle to reliably conduct fundamental fact-checking tasks, such as identifying whether an image is authentic. Each model we tested misidentified at least one real photo as AI-generated.
August 26, 2025 at 8:46 PM
However, they also struggle to reliably conduct fundamental fact-checking tasks, such as identifying whether an image is authentic. Each model we tested misidentified at least one real photo as AI-generated.
We observed that errors compounded in the LLMs’ reasoning: if one conclusion was wrong, subsequent steps often built on that error.
August 26, 2025 at 8:46 PM
We observed that errors compounded in the LLMs’ reasoning: if one conclusion was wrong, subsequent steps often built on that error.
For example, when we asked Grok Deep Research to identify the provenance of a photo taken during flooding in Valencia, Spain, it fixated on text on person’s shirt that said “Venice Beach," leading it to incorrectly conclude that the photo was taken during flooding in Venice, Italy.
August 26, 2025 at 8:46 PM
For example, when we asked Grok Deep Research to identify the provenance of a photo taken during flooding in Valencia, Spain, it fixated on text on person’s shirt that said “Venice Beach," leading it to incorrectly conclude that the photo was taken during flooding in Venice, Italy.
NEW in @columjournreview.bsky.social: Why AI models are bad at fact-checking photos www.cjr.org/tow_center/w...
August 26, 2025 at 8:12 PM
NEW in @columjournreview.bsky.social: Why AI models are bad at fact-checking photos www.cjr.org/tow_center/w...
lol came across this amazing interaction on X related to my + @aisvarya17.bsky.social's research
April 21, 2025 at 12:35 AM
lol came across this amazing interaction on X related to my + @aisvarya17.bsky.social's research
5️⃣ Finally, content licensing deals with news sources provided no guarantee of accurate citation in chatbot responses. For instance, @sfchronicle.com has a deal with OpenAI, but ChatGPT only correctly identified 1 of the 10 excerpts we shared from the publisher.
March 6, 2025 at 2:50 PM
5️⃣ Finally, content licensing deals with news sources provided no guarantee of accurate citation in chatbot responses. For instance, @sfchronicle.com has a deal with OpenAI, but ChatGPT only correctly identified 1 of the 10 excerpts we shared from the publisher.
4️⃣ Generative search tools made up links and cited syndicated and copied versions of articles. For example, more than half of responses from Gemini and Grok-3 cited fabricated or broken URLs that led to error pages
March 6, 2025 at 2:50 PM
4️⃣ Generative search tools made up links and cited syndicated and copied versions of articles. For example, more than half of responses from Gemini and Grok-3 cited fabricated or broken URLs that led to error pages
3️⃣ Multiple chatbots seemed to bypass robots.txt preferences. For instance, Perplexity correctly identified all 10 excerpts from paywalled articles we shared from National Geographic even though the publisher has disallowed Perplexity's crawler and has no formal relationship with the company.
March 6, 2025 at 2:50 PM
3️⃣ Multiple chatbots seemed to bypass robots.txt preferences. For instance, Perplexity correctly identified all 10 excerpts from paywalled articles we shared from National Geographic even though the publisher has disallowed Perplexity's crawler and has no formal relationship with the company.
2️⃣ While premium chatbots like Grok-3 and Perplexity answered more prompts correctly than their corresponding free equivalents, they paradoxically also provided more confidently incorrect answers than their free counterparts.
March 6, 2025 at 2:50 PM
2️⃣ While premium chatbots like Grok-3 and Perplexity answered more prompts correctly than their corresponding free equivalents, they paradoxically also provided more confidently incorrect answers than their free counterparts.
We found that:
1️⃣ Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.
1️⃣ Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.
March 6, 2025 at 2:50 PM
We found that:
1️⃣ Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.
1️⃣ Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.
We chose 20 publishers with varying stances on AI access, randomly selected 10 articles from each publisher and manually chose excerpts from the articles to use in our queries. We asked each chatbot to identify the corresponding article’s headline, original publisher, publication date and URL.
March 6, 2025 at 2:50 PM
We chose 20 publishers with varying stances on AI access, randomly selected 10 articles from each publisher and manually chose excerpts from the articles to use in our queries. We asked each chatbot to identify the corresponding article’s headline, original publisher, publication date and URL.
Really excellent explainer by @shaynelongpre.bsky.social that clearly lays out what's at stake in the "AI crawler wars"
February 21, 2025 at 4:56 PM
Really excellent explainer by @shaynelongpre.bsky.social that clearly lays out what's at stake in the "AI crawler wars"
For @columjournreview.bsky.social, @aisvarya17.bsky.social and I tested Grok-3's ability to identify quotes from news publishers. Out of the 200 quotes we tested, 102 returned hallucinated citations and the original source article was correctly cited only nine times. www.cjr.org/the_media_to...
February 20, 2025 at 2:51 PM
For @columjournreview.bsky.social, @aisvarya17.bsky.social and I tested Grok-3's ability to identify quotes from news publishers. Out of the 200 quotes we tested, 102 returned hallucinated citations and the original source article was correctly cited only nine times. www.cjr.org/the_media_to...
Excellent presentation at #IASEAI25 today by @juliaangwin.com, who stressed the importance of putting pressure on AI companies to prioritize accuracy and truth in their outputs (in this photo she’s citing research by the great @sayash.bsky.social and @randomwalker.bsky.social
February 7, 2025 at 2:54 PM
Excellent presentation at #IASEAI25 today by @juliaangwin.com, who stressed the importance of putting pressure on AI companies to prioritize accuracy and truth in their outputs (in this photo she’s citing research by the great @sayash.bsky.social and @randomwalker.bsky.social
"It’s a 'you' problem that TikTok will happily serve up joke videos until the moment of your death."
January 8, 2025 at 9:30 PM
"It’s a 'you' problem that TikTok will happily serve up joke videos until the moment of your death."
Ultimately, what we found is that ChatGPT search offers publishers the illusion of control. No publisher – regardless of degree of affiliation with OpenAI – was spared from inaccurate representations of its content. (7/9)
November 27, 2024 at 7:36 PM
Ultimately, what we found is that ChatGPT search offers publishers the illusion of control. No publisher – regardless of degree of affiliation with OpenAI – was spared from inaccurate representations of its content. (7/9)
The search tool's temperature settings likely contributed to inconsistent outputs. When we asked it to identify the same Washington Post quote twice, it answered incorrectly on one occasion and correctly on another. (6/9)
November 27, 2024 at 7:34 PM
The search tool's temperature settings likely contributed to inconsistent outputs. When we asked it to identify the same Washington Post quote twice, it answered incorrectly on one occasion and correctly on another. (6/9)
In some instances, the search bot attributed the text to syndicated or plagiarized versions of the articles, instead of the original source. (5/9)
November 27, 2024 at 7:33 PM
In some instances, the search bot attributed the text to syndicated or plagiarized versions of the articles, instead of the original source. (5/9)
More than a third of ChatGPT’s responses to our queries included incorrect citations such as the one below, where it attributes a letter to the editor published in the @orlandosentinel.com to TIME (4/9)
November 27, 2024 at 7:32 PM
More than a third of ChatGPT’s responses to our queries included incorrect citations such as the one below, where it attributes a letter to the editor published in the @orlandosentinel.com to TIME (4/9)
Specifically, we were looking to see if it accurately returned the publisher, date and URL of the quotes we shared.
Of the 200 responses to our queries, 153 were either partially or entirely incorrect. But the chatbot only expressed a lack of confidence in its answers 7 times. (3/9)
Of the 200 responses to our queries, 153 were either partially or entirely incorrect. But the chatbot only expressed a lack of confidence in its answers 7 times. (3/9)
November 27, 2024 at 7:31 PM
Specifically, we were looking to see if it accurately returned the publisher, date and URL of the quotes we shared.
Of the 200 responses to our queries, 153 were either partially or entirely incorrect. But the chatbot only expressed a lack of confidence in its answers 7 times. (3/9)
Of the 200 responses to our queries, 153 were either partially or entirely incorrect. But the chatbot only expressed a lack of confidence in its answers 7 times. (3/9)
Meanwhile, critics warn that this farsighted movement is drawing attention and resources away from urgent, presently observable harms of AI deployments.
October 30, 2023 at 9:51 PM
Meanwhile, critics warn that this farsighted movement is drawing attention and resources away from urgent, presently observable harms of AI deployments.