Institute for Replication
@i4replication.bsky.social
The Institute for Replication (I4R) works to improve the credibility of science by promoting and conducting reproductions and replications.
i4replication.org
i4replication.org
We won't be releasing any additional info on social media until the situation is resolved with lawyers, but will keep releasing reports through our DP series. Here are two 40-page reports completed earlier this year.
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
October 30, 2025 at 1:41 PM
We won't be releasing any additional info on social media until the situation is resolved with lawyers, but will keep releasing reports through our DP series. Here are two 40-page reports completed earlier this year.
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
econpapers.repec.org/paper/zbwi4r...
We are hosting virtual Replication Games on Friday November 13th, 2025 with @ukrepro.bsky.social ro.bsky.social. This is our 2nd year collaborating with UKRN. Psych, public health, pol sci and econ studies will be reproduced!
Register here: www.surveymonkey.ca/r/I4R_Replic...
Register here: www.surveymonkey.ca/r/I4R_Replic...
October 20, 2025 at 6:21 PM
We are hosting virtual Replication Games on Friday November 13th, 2025 with @ukrepro.bsky.social ro.bsky.social. This is our 2nd year collaborating with UKRN. Psych, public health, pol sci and econ studies will be reproduced!
Register here: www.surveymonkey.ca/r/I4R_Replic...
Register here: www.surveymonkey.ca/r/I4R_Replic...
We are hosting Replication Games at the 95th Meeting of the Southern Economics Association on Friday November 21st, 2025. This is a new collaboration with the SEA!
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
October 15, 2025 at 12:55 PM
We are hosting Replication Games at the 95th Meeting of the Southern Economics Association on Friday November 21st, 2025. This is a new collaboration with the SEA!
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
We are hosting this year's MAER-Net Colloquium on Friday and Saturday. The program and more information are available here: www.maer-net.org/2025-ottawa
Plenary sessions (with Andrew Gelman, Shinichi Nakagawa and others) and some parallel sessions will be live-streamed for free
Plenary sessions (with Andrew Gelman, Shinichi Nakagawa and others) and some parallel sessions will be live-streamed for free
October 15, 2025 at 12:45 PM
We are hosting this year's MAER-Net Colloquium on Friday and Saturday. The program and more information are available here: www.maer-net.org/2025-ottawa
Plenary sessions (with Andrew Gelman, Shinichi Nakagawa and others) and some parallel sessions will be live-streamed for free
Plenary sessions (with Andrew Gelman, Shinichi Nakagawa and others) and some parallel sessions will be live-streamed for free
Big thanks to our research scientist, Derek Mikola, for organizing 4 games over the past 2 weeks. Lots of planes/trains from Galway, to Berlin, Paris and now Lyon. Today's event in Lyon is underway!
October 9, 2025 at 12:08 PM
Big thanks to our research scientist, Derek Mikola, for organizing 4 games over the past 2 weeks. Lots of planes/trains from Galway, to Berlin, Paris and now Lyon. Today's event in Lyon is underway!
We also document changes in language about the randomization between the working paper and published version, and inconsistencies in the authors’ response. See full report (here osf.io/wp8fr/) for more details.
August 27, 2025 at 3:33 PM
We also document changes in language about the randomization between the working paper and published version, and inconsistencies in the authors’ response. See full report (here osf.io/wp8fr/) for more details.
the opposite—that the study neither involved nor required experimental group assignment.
August 27, 2025 at 3:33 PM
the opposite—that the study neither involved nor required experimental group assignment.
The data show that group assignment was inherited from the retracted EER study --> they use a subset of the data. Although both studies state that randomization occurred at the household level, the treat variable in the replication data is constant within villages. See figure 2.
August 27, 2025 at 3:33 PM
The data show that group assignment was inherited from the retracted EER study --> they use a subset of the data. Although both studies state that randomization occurred at the household level, the treat variable in the replication data is constant within villages. See figure 2.
In our report, we challenge the claim in Begum et al. (2018) that HHs were assigned randomly to treatment (allocation tasks). Because replication data is unavailable, we rely on replication data from a follow-up study (Begum et al. 2022). More on this other study later on.
August 27, 2025 at 3:33 PM
In our report, we challenge the claim in Begum et al. (2018) that HHs were assigned randomly to treatment (allocation tasks). Because replication data is unavailable, we rely on replication data from a follow-up study (Begum et al. 2022). More on this other study later on.
#GDRI_rep Update 8a: Major updates coming up over the next few weeks.
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
August 27, 2025 at 3:33 PM
#GDRI_rep Update 8a: Major updates coming up over the next few weeks.
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev.
Additional studies are connected (see diagram)
🧵
The Tilburg Replication Games are over! We had lots of fun and are looking forward to the TIBER symposium starting tomorrow.
www.tilburguniversity.edu/research/ins...
www.tilburguniversity.edu/research/ins...
August 22, 2025 at 1:42 AM
The Tilburg Replication Games are over! We had lots of fun and are looking forward to the TIBER symposium starting tomorrow.
www.tilburguniversity.edu/research/ins...
www.tilburguniversity.edu/research/ins...
Only a one week left left to register to the Brno Replication Games on Sept 7th. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
August 13, 2025 at 2:35 PM
Only a one week left left to register to the Brno Replication Games on Sept 7th. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
We have our first games ever in Ireland next month. The event will be at the University of Galway as part of CERIS’s Annual Workshop on September 26th. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
August 6, 2025 at 2:34 PM
We have our first games ever in Ireland next month. The event will be at the University of Galway as part of CERIS’s Annual Workshop on September 26th. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
We are collaborating with the @eeanews.bsky.social for a third year in a row. EEA Replication Games in Bordeaux, France on August 24. Virtual participation is possible and the focus is on economics papers for this event!
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
July 28, 2025 at 1:37 PM
We are collaborating with the @eeanews.bsky.social for a third year in a row. EEA Replication Games in Bordeaux, France on August 24. Virtual participation is possible and the focus is on economics papers for this event!
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
Only a couple days left to register to the Tilburg Replication Games on August 21. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
July 25, 2025 at 1:30 PM
Only a couple days left to register to the Tilburg Replication Games on August 21. Virtual participation is possible and coauthorship to a meta paper is granted.
Register here: www.surveymonkey.ca/r/Replicatio...
Register here: www.surveymonkey.ca/r/Replicatio...
We have 8 replication games over the next 13 weeks (Tilburg, Bordeaux, Brno, Galway, Berlin, Paris, Lyon and Ottawa)! Not too late to register to any of these events:
i4replication.org/games.html
i4replication.org/games.html
July 23, 2025 at 12:32 PM
We have 8 replication games over the next 13 weeks (Tilburg, Bordeaux, Brno, Galway, Berlin, Paris, Lyon and Ottawa)! Not too late to register to any of these events:
i4replication.org/games.html
i4replication.org/games.html
Of note, the editorial board at JPopE had decided not to retract the article. They instead invited the original author to write a new article. So the retraction by Springer goes against the initial decision of editors. The original author disagrees with this retraction.
July 21, 2025 at 1:31 PM
Of note, the editorial board at JPopE had decided not to retract the article. They instead invited the original author to write a new article. So the retraction by Springer goes against the initial decision of editors. The original author disagrees with this retraction.
Meanwhile, the publisher (Springer) decided to retract the original article. The decision is from Springer, possibly following emails from the replicators to COPE (Committee on Publication Ethics).
link.springer.com/article/10.1...
link.springer.com/article/10.1...
July 21, 2025 at 1:31 PM
Meanwhile, the publisher (Springer) decided to retract the original article. The decision is from Springer, possibly following emails from the replicators to COPE (Committee on Publication Ethics).
link.springer.com/article/10.1...
link.springer.com/article/10.1...
The author then responded to the replicators' response. See page 13 (ideas.repec.org/p/zbw/i4rdps...) of the author's response.
July 21, 2025 at 1:31 PM
The author then responded to the replicators' response. See page 13 (ideas.repec.org/p/zbw/i4rdps...) of the author's response.
As per protocol, we checked the tone and offered the replicators the opportunity to respond. Their short response is here: ideas.repec.org/p/zbw/i4rdps....
July 21, 2025 at 1:31 PM
As per protocol, we checked the tone and offered the replicators the opportunity to respond. Their short response is here: ideas.repec.org/p/zbw/i4rdps....
We were contacted by the original author mid-June. He requested to release a response as an I4R discussion paper. (He had initially declined our invitation to respond, pointing out his re-analysis published by JPopE as a response.)
ideas.repec.org/p/zbw/i4rdps...
ideas.repec.org/p/zbw/i4rdps...
July 21, 2025 at 1:31 PM
We were contacted by the original author mid-June. He requested to release a response as an I4R discussion paper. (He had initially declined our invitation to respond, pointing out his re-analysis published by JPopE as a response.)
ideas.repec.org/p/zbw/i4rdps...
ideas.repec.org/p/zbw/i4rdps...
Last opportunity to reproduce/replicate a study published @nathumbehav.nature.com. This is an official collaboration with NHB. All authors of completed reports will be included as coauthors to our meta-paper to be submitted to NHB this Fall.
🧵
🧵
July 3, 2025 at 12:40 PM
Last opportunity to reproduce/replicate a study published @nathumbehav.nature.com. This is an official collaboration with NHB. All authors of completed reports will be included as coauthors to our meta-paper to be submitted to NHB this Fall.
🧵
🧵
Only a few days left to register to the Amsterdam Replication Games on July 19. Virtual participation is possible and coauthorship to a meta paper is granted.
Register: www.surveymonkey.ca/r/Replicatio...
Register: www.surveymonkey.ca/r/Replicatio...
July 1, 2025 at 11:21 AM
Only a few days left to register to the Amsterdam Replication Games on July 19. Virtual participation is possible and coauthorship to a meta paper is granted.
Register: www.surveymonkey.ca/r/Replicatio...
Register: www.surveymonkey.ca/r/Replicatio...
Datasets were often misplaced/archived in obsolete formats, or even physically damaged (by rodents🐀, a barrier largely a thing of the past)
Journal policies not requiring data retention beyond 20 years
Ownership of the data belonging to a separate entity that the author was not affiliated with...
Journal policies not requiring data retention beyond 20 years
Ownership of the data belonging to a separate entity that the author was not affiliated with...
May 29, 2025 at 7:06 PM
Datasets were often misplaced/archived in obsolete formats, or even physically damaged (by rodents🐀, a barrier largely a thing of the past)
Journal policies not requiring data retention beyond 20 years
Ownership of the data belonging to a separate entity that the author was not affiliated with...
Journal policies not requiring data retention beyond 20 years
Ownership of the data belonging to a separate entity that the author was not affiliated with...
As he puts it "it did not go great". Out of 115 authors we contacted, we received responses from just 10. These 10 authors were involved in 41 of the 182 papers.
We didn't hear back from any authors of papers on seasonal malaria chemoprevention (SMC) or bednets (LLINs).
We didn't hear back from any authors of papers on seasonal malaria chemoprevention (SMC) or bednets (LLINs).
May 29, 2025 at 7:06 PM
As he puts it "it did not go great". Out of 115 authors we contacted, we received responses from just 10. These 10 authors were involved in 41 of the 182 papers.
We didn't hear back from any authors of papers on seasonal malaria chemoprevention (SMC) or bednets (LLINs).
We didn't hear back from any authors of papers on seasonal malaria chemoprevention (SMC) or bednets (LLINs).