Essi Kujansuu
@essieconomist.bsky.social
Experimental economist. Post-doc at Innsbruck, visiting researcher at Turku, PhD from EUI.
https://sites.google.com/view/essi-kujansuu/home
https://sites.google.com/view/essi-kujansuu/home
Reposted by Essi Kujansuu
#GDRI_rep Update 8a: Major updates coming up over the next few weeks.
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
August 27, 2025 at 3:33 PM
#GDRI_rep Update 8a: Major updates coming up over the next few weeks.
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
Reposted by Essi Kujansuu
GiveWell sought an independent check on the evidence supporting >$1 billion it has granted to its top charities. This fall, Alex Cohen asked whether we could obtain the datasets needed to reproduce the underlying RCTs. We said yes; charging only a modest fee to cover our time.
🧵 + 🐀 + blog post!
🧵 + 🐀 + blog post!
May 29, 2025 at 7:06 PM
GiveWell sought an independent check on the evidence supporting >$1 billion it has granted to its top charities. This fall, Alex Cohen asked whether we could obtain the datasets needed to reproduce the underlying RCTs. We said yes; charging only a modest fee to cover our time.
🧵 + 🐀 + blog post!
🧵 + 🐀 + blog post!
Reposted by Essi Kujansuu
#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.
Short 🧵
Short 🧵
March 27, 2025 at 9:48 PM
#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.
Short 🧵
Short 🧵
Reposted by Essi Kujansuu
I am very happy that the editors took our report seriously. Hope more retractions will follow.
#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.
Short 🧵
Short 🧵
March 27, 2025 at 9:52 PM
I am very happy that the editors took our report seriously. Hope more retractions will follow.
You have to respect a response to a re-response to a response to a report.
Our full report and our codes are here: osf.io/5k4f7/files/.... This OSF page also includes the authors' response, our response to their response and a second authors response to our response to their response.
We thank the authors for taking the time to respond and this back and forth.
We thank the authors for taking the time to respond and this back and forth.
March 14, 2025 at 1:27 PM
You have to respect a response to a re-response to a response to a report.
Reposted by Essi Kujansuu
Our full report and our codes are here: osf.io/5k4f7/files/.... This OSF page also includes the authors' response, our response to their response and a second authors response to our response to their response.
We thank the authors for taking the time to respond and this back and forth.
We thank the authors for taking the time to respond and this back and forth.
March 14, 2025 at 1:25 PM
Our full report and our codes are here: osf.io/5k4f7/files/.... This OSF page also includes the authors' response, our response to their response and a second authors response to our response to their response.
We thank the authors for taking the time to respond and this back and forth.
We thank the authors for taking the time to respond and this back and forth.
Reposted by Essi Kujansuu
#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.
March 14, 2025 at 1:25 PM
#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.
Reposted by Essi Kujansuu
#GDRI_rep Update 3: The paper "Improving Hygiene and Sanitation through Parental Skill Training" by Asadul Islam, Umair Khalil and Tabassum Rahman was conditionally accepted at Economic Development and Cultural Change. We requested a replication package. It has now been withdrawn
March 9, 2025 at 9:05 PM
#GDRI_rep Update 3: The paper "Improving Hygiene and Sanitation through Parental Skill Training" by Asadul Islam, Umair Khalil and Tabassum Rahman was conditionally accepted at Economic Development and Cultural Change. We requested a replication package. It has now been withdrawn
Reposted by Essi Kujansuu
We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.
#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/
March 4, 2025 at 12:44 AM
We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.
#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/
March 3, 2025 at 6:56 PM
We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.
Reposted by Essi Kujansuu
The editor-in-chief at Food Policy (@_Food_Policy) contacted us because an author requested to withdraw their name for: “Determinants and Dynamics of Food Insecurity During COVID-19 in Rural Bangladesh”. We explained the situation and requested a replication package.
March 3, 2025 at 1:48 AM
The editor-in-chief at Food Policy (@_Food_Policy) contacted us because an author requested to withdraw their name for: “Determinants and Dynamics of Food Insecurity During COVID-19 in Rural Bangladesh”. We explained the situation and requested a replication package.
Reposted by Essi Kujansuu
We completed a full report on “Food insecurity and mental health of women during COVID-19: Evidence from a developing country” PLOS One and shared it with the authors and journal. Two out of three authors have already requested to withdraw their names.
March 3, 2025 at 1:48 AM
We completed a full report on “Food insecurity and mental health of women during COVID-19: Evidence from a developing country” PLOS One and shared it with the authors and journal. Two out of three authors have already requested to withdraw their names.
Reposted by Essi Kujansuu
Here is a list of studies, with links to I4R reports, authors’ responses/statements, and emails sent to the editors and authors (with dates): osf.io/23x64/.
This excel document is being updated every day.
This excel document is being updated every day.
March 3, 2025 at 1:48 AM
Here is a list of studies, with links to I4R reports, authors’ responses/statements, and emails sent to the editors and authors (with dates): osf.io/23x64/.
This excel document is being updated every day.
This excel document is being updated every day.
Reposted by Essi Kujansuu
We are now aiming to reproduce about 20 published studies. This number keeps increasing as new GDRI studies are forthcoming or are now conditionally accepted.
March 3, 2025 at 1:48 AM
We are now aiming to reproduce about 20 published studies. This number keeps increasing as new GDRI studies are forthcoming or are now conditionally accepted.
Reposted by Essi Kujansuu
#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵
March 3, 2025 at 1:48 AM
#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵
Reposted by Essi Kujansuu
I4R doing the lord’s work. Hopefully these will be a collective wake up call to be more careful and transparent about complicated studies in applied micro (and elsewhere too)
Let’s take a moment to thank all our amazing replicators: Jörg Ankel-Peters, Juan Pablo Aparicio, Gunther Bensch, Carl Bonander, Nikolai Cook, Lenka Fiala, Jack Fitzgerald, Olle Hammar, Felix Holzmeister, Niklas Jakobsson, Anders Kjelsrud, Andreas Kotsadam, Essi Kujansuu, ...
February 24, 2025 at 5:20 PM
I4R doing the lord’s work. Hopefully these will be a collective wake up call to be more careful and transparent about complicated studies in applied micro (and elsewhere too)
Reposted by Essi Kujansuu
While the (quasi-) experimental methods increase credibility of causal results, there are two related concerns that make studies hard to replicate:
1. decline in reporting of null results
2. increase in use of private sector data
a mini thread 🧵based on work with @trfetzer.com
1. decline in reporting of null results
2. increase in use of private sector data
a mini thread 🧵based on work with @trfetzer.com
February 24, 2025 at 11:05 AM
While the (quasi-) experimental methods increase credibility of causal results, there are two related concerns that make studies hard to replicate:
1. decline in reporting of null results
2. increase in use of private sector data
a mini thread 🧵based on work with @trfetzer.com
1. decline in reporting of null results
2. increase in use of private sector data
a mini thread 🧵based on work with @trfetzer.com
Reposted by Essi Kujansuu
Massive team work in replicating papers and cross checking samples and treatments across multiple papers — what a massive undertaking!
This is evidence that journals *must* require data and code sharing.
I’m wondering if anything in the peer review process could have prevented this
This is evidence that journals *must* require data and code sharing.
I’m wondering if anything in the peer review process could have prevented this
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵
February 24, 2025 at 7:37 PM
Massive team work in replicating papers and cross checking samples and treatments across multiple papers — what a massive undertaking!
This is evidence that journals *must* require data and code sharing.
I’m wondering if anything in the peer review process could have prevented this
This is evidence that journals *must* require data and code sharing.
I’m wondering if anything in the peer review process could have prevented this
Reposted by Essi Kujansuu
If anything is clear from the last few days, it's surely that Ukraine belongs in the EU
February 24, 2025 at 6:25 PM
If anything is clear from the last few days, it's surely that Ukraine belongs in the EU
Reposted by Essi Kujansuu
Wow. This is bigger than I thought.
We should be involved in the data collection as much as we are in the data analysis. Well done @i4replication.bsky.social!
We should be involved in the data collection as much as we are in the data analysis. Well done @i4replication.bsky.social!
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵
February 24, 2025 at 5:07 PM
Wow. This is bigger than I thought.
We should be involved in the data collection as much as we are in the data analysis. Well done @i4replication.bsky.social!
We should be involved in the data collection as much as we are in the data analysis. Well done @i4replication.bsky.social!
Reposted by Essi Kujansuu
Disregarding the sadness of this story, it’s been great working with this awesome team! @jrgptrs.bsky.social @fialalenka.bsky.social @jackfitzgerald.bsky.social @flxhlzmstr.bsky.social @nikljako.bsky.social @anderskjelsrud.bsky.social @kotsadam.bsky.social @essieconomist.bsky.social
HUGE thanks to our 15+ replicators who have been working on this project for 3 months now. They are not paid, and this is all pro bono. Their work should get published as comments! It has been a pleasure working with you all. And somehow this feels like the beginning!
February 24, 2025 at 3:50 PM
Disregarding the sadness of this story, it’s been great working with this awesome team! @jrgptrs.bsky.social @fialalenka.bsky.social @jackfitzgerald.bsky.social @flxhlzmstr.bsky.social @nikljako.bsky.social @anderskjelsrud.bsky.social @kotsadam.bsky.social @essieconomist.bsky.social
Reposted by Essi Kujansuu
The craziest thing about it is that if you want to believe the most “positive” interpretation, you have to agree that they’re bad at RCTs. Messy, don’t follow protocols, etc.
Yet they consistently get the biggest effect sizes!
It’s almost like a proof by contradiction.
Yet they consistently get the biggest effect sizes!
It’s almost like a proof by contradiction.
February 24, 2025 at 5:49 AM
The craziest thing about it is that if you want to believe the most “positive” interpretation, you have to agree that they’re bad at RCTs. Messy, don’t follow protocols, etc.
Yet they consistently get the biggest effect sizes!
It’s almost like a proof by contradiction.
Yet they consistently get the biggest effect sizes!
It’s almost like a proof by contradiction.
Reposted by Essi Kujansuu
This effort was only possible because these journals require data and code repositories. This allowed us to compare data across papers and identify undisclosed connections between 15 studies (e.g., overlapping samples, overlapping interventions, reused treatment assignment).
February 24, 2025 at 1:55 AM
This effort was only possible because these journals require data and code repositories. This allowed us to compare data across papers and identify undisclosed connections between 15 studies (e.g., overlapping samples, overlapping interventions, reused treatment assignment).
Reposted by Essi Kujansuu
As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵
February 24, 2025 at 6:42 AM
As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.
Reposted by Essi Kujansuu
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵
February 24, 2025 at 1:55 AM
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵