Rick Carlsson
@rickcarlsson.bsky.social
Open Science. Meta-Science. Editor-in-Chief at Meta-Psychology.
Reposted by Rick Carlsson
I am hoping to recruit a graduate student for next year. That person would help conduct research on leadership, individual differences, and methodological skullduggery. Please forward to any potentially interested students.
November 17, 2025 at 6:13 PM
I am hoping to recruit a graduate student for next year. That person would help conduct research on leadership, individual differences, and methodological skullduggery. Please forward to any potentially interested students.
Reposted by Rick Carlsson
My Shiny app containing 3530 Open Science blog posts discussing the replication crisis is updated - you can now use the SEARCH box. I fixed it as my new PhD Julia wanted to know who had called open scientists 'Methodological Terrorists' :) shiny.ieis.tue.nl/open_science...
Open Science Blog Browser
Open Science Blog Browser
shiny.ieis.tue.nl
November 8, 2025 at 7:15 PM
My Shiny app containing 3530 Open Science blog posts discussing the replication crisis is updated - you can now use the SEARCH box. I fixed it as my new PhD Julia wanted to know who had called open scientists 'Methodological Terrorists' :) shiny.ieis.tue.nl/open_science...
Reposted by Rick Carlsson
I struggle to think of a statistical "method" that has been more damaging to psychology (and related disciplines) than mediation analysis based on observational data. IMHO all such manuscripts should be desk rejected.
November 9, 2025 at 2:59 PM
I struggle to think of a statistical "method" that has been more damaging to psychology (and related disciplines) than mediation analysis based on observational data. IMHO all such manuscripts should be desk rejected.
Reposted by Rick Carlsson
This one surely has something on offer for every one: In this cross-sectional mediation analysis, the "effects" of soft drink consumption on depression were "mediated" by abundance of Eggerthela in the gut microbiome.
This was sent to me via dm and now you all got to suffer as well.
This was sent to me via dm and now you all got to suffer as well.
Soft Drink Consumption and Depression Mediated by Gut Microbiome Alterations
This cohort study examines the association between soft drink consumption and major depressive disorder diagnosis and severity and whether this association is mediated by changes in the gut microbiota...
jamanetwork.com
November 9, 2025 at 7:19 AM
This one surely has something on offer for every one: In this cross-sectional mediation analysis, the "effects" of soft drink consumption on depression were "mediated" by abundance of Eggerthela in the gut microbiome.
This was sent to me via dm and now you all got to suffer as well.
This was sent to me via dm and now you all got to suffer as well.
Reposted by Rick Carlsson
New blog post! Calculating a correlation is easy enough. But let's say you calculated two of them and they happen to differ. What follows from that? Turns out there are too many moving parts for an easy answer.
www.the100.ci/2025/07/28/w...
www.the100.ci/2025/07/28/w...
What’s in a correlation?
Correlation may not imply causation, but let’s just ignore that for a second. Correlations are standardized effect size metrics and as such have some quirks by design. These are benign enough when you...
www.the100.ci
July 28, 2025 at 12:21 PM
New blog post! Calculating a correlation is easy enough. But let's say you calculated two of them and they happen to differ. What follows from that? Turns out there are too many moving parts for an easy answer.
www.the100.ci/2025/07/28/w...
www.the100.ci/2025/07/28/w...
Reposted by Rick Carlsson
I feel like psychometrics is uniquely bad at naming things. Parallel, congeneric, tau-equivalent, essentially tau-equivalent measures? Configural, metric, scalar, residual invariance? Item difficulty defined so that the higher the difficulty, the easier the item???
November 7, 2025 at 5:45 PM
I feel like psychometrics is uniquely bad at naming things. Parallel, congeneric, tau-equivalent, essentially tau-equivalent measures? Configural, metric, scalar, residual invariance? Item difficulty defined so that the higher the difficulty, the easier the item???
Reposted by Rick Carlsson
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-... where I reflect on how we should check the quality of novel statistical techniques.
October 29, 2025 at 7:39 AM
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-... where I reflect on how we should check the quality of novel statistical techniques.
Reposted by Rick Carlsson
Pro tip: When fabricating a dataset, also make several errors in the analysis of the fake data. Then when sleuths point out the errors as well as the fabrication, you can make a more plausible fake dataset when the editor lets you correct the statistical errors "in the interest of fairness".
October 31, 2025 at 8:56 PM
Pro tip: When fabricating a dataset, also make several errors in the analysis of the fake data. Then when sleuths point out the errors as well as the fabrication, you can make a more plausible fake dataset when the editor lets you correct the statistical errors "in the interest of fairness".
Reposted by Rick Carlsson
After a day or two of sleuthing I have realized that a recent (and likely to be very influential) meta-analysis missed 30%-40% of the relevant literature. Do I bother trying to correct the record?
June 26, 2025 at 11:45 PM
After a day or two of sleuthing I have realized that a recent (and likely to be very influential) meta-analysis missed 30%-40% of the relevant literature. Do I bother trying to correct the record?
Reposted by Rick Carlsson
Psych-DS is (1) spellcheck for your datasets and (2) a pathway to standardizing data in our academic fields that *everyone* can learn.
And it's live RIGHT NOW!
psych-ds.github.io
(This is the announcement post I've been leading up to)
And it's live RIGHT NOW!
psych-ds.github.io
(This is the announcement post I've been leading up to)
Psych-DS
A specification for psychological datasets. JSON metadata, predictable directory structure, and machine-readable specifications for tabular datasets.
psych-ds.github.io
April 9, 2025 at 7:37 PM
Psych-DS is (1) spellcheck for your datasets and (2) a pathway to standardizing data in our academic fields that *everyone* can learn.
And it's live RIGHT NOW!
psych-ds.github.io
(This is the announcement post I've been leading up to)
And it's live RIGHT NOW!
psych-ds.github.io
(This is the announcement post I've been leading up to)
Reposted by Rick Carlsson
This article notes that "Among 40 papers flagged as having issues, he [i.e., me] found 14 false positives (for example, the model stating that a figure referred to in the text did not appear in the paper, when it did)".
For background, here are the cases that I believe to be false positives.
/0
For background, here are the cases that I believe to be false positives.
/0
Is everyone huffing paint?
Crypto guy claims to have built an LLM-based tool to detect errors in research papers; funded using its own cryptocurrency; will let coin holders choose what papers to go after; it's unvetted and a total black box—and Nature reports it as if it's a new protein structure.
Crypto guy claims to have built an LLM-based tool to detect errors in research papers; funded using its own cryptocurrency; will let coin holders choose what papers to go after; it's unvetted and a total black box—and Nature reports it as if it's a new protein structure.
AI tools are spotting errors in research papers: inside a growing movement
Study that hyped the toxicity of black plastic utensils inspires projects that use large language models to check papers.
www.nature.com
March 10, 2025 at 9:32 PM
This article notes that "Among 40 papers flagged as having issues, he [i.e., me] found 14 false positives (for example, the model stating that a figure referred to in the text did not appear in the paper, when it did)".
For background, here are the cases that I believe to be false positives.
/0
For background, here are the cases that I believe to be false positives.
/0
Reposted by Rick Carlsson
OSIRIS (Open Science to Improve Reproducibility in Science) is currently conducting several interventional studies on Open Science practices. We invite interested colleagues to register and join a group that may take part in these studies. More info and registration 👇
Join the OSIRIS Reproducibility Network: A Collaborative Effort to Advance Open Science
Become part of a dynamic network committed to enhancing research transparency and reproducibility through innovative randomized controlled trials. Contribute your expertise, gain valuable training, an...
osiris4r.eu
March 6, 2025 at 2:06 PM
OSIRIS (Open Science to Improve Reproducibility in Science) is currently conducting several interventional studies on Open Science practices. We invite interested colleagues to register and join a group that may take part in these studies. More info and registration 👇
Reposted by Rick Carlsson
I will therefore not complain if someone tries to introduce more structure. And so here you go: a decent attempt at methods building from @jamesheathers.bsky.social :
jamesheathers.curve.space
(Could I have a PDF though? ) 2/
jamesheathers.curve.space
(Could I have a PDF though? ) 2/
An Introduction To Forensic Metascience
jamesheathers.curve.space
February 15, 2025 at 12:35 PM
I will therefore not complain if someone tries to introduce more structure. And so here you go: a decent attempt at methods building from @jamesheathers.bsky.social :
jamesheathers.curve.space
(Could I have a PDF though? ) 2/
jamesheathers.curve.space
(Could I have a PDF though? ) 2/
Reposted by Rick Carlsson
Wow, this should be super interesting.
Wait, no, that's his book from 2022. Sorry!
Wait, no, that's his book from 2022. Sorry!
February 24, 2025 at 10:02 AM
Wow, this should be super interesting.
Wait, no, that's his book from 2022. Sorry!
Wait, no, that's his book from 2022. Sorry!
Reposted by Rick Carlsson
I am sorry that you fail to see the benefit of this paper or this type of work, Brett. Read it first, perhaps.
January 10, 2025 at 9:11 PM
I am sorry that you fail to see the benefit of this paper or this type of work, Brett. Read it first, perhaps.
Reposted by Rick Carlsson
We are excited to announce the review of Lades et al. (2020) "Daily emotional well-being during the COVID-19 pandemic". Based on the review by Joop Adema, we find Moderate Errors which do not affect the core conclusions. We recommend the authors seek a correction error.reviews/reviews/lade...
Lades et al. (2020)
ERROR is a bug bounty program for science to systematically detect and report errors in academic publications
error.reviews
January 8, 2025 at 12:34 PM
We are excited to announce the review of Lades et al. (2020) "Daily emotional well-being during the COVID-19 pandemic". Based on the review by Joop Adema, we find Moderate Errors which do not affect the core conclusions. We recommend the authors seek a correction error.reviews/reviews/lade...
Reposted by Rick Carlsson
Hey Quant Folks: Does anyone have any recommendations for testing if two correlation matrices are meaningfully and/or statistically significantly different from each other?
NB: In the words of Jeremy Irons: "Please feel free to speak to me as you might to a young child or a golden retriever."
NB: In the words of Jeremy Irons: "Please feel free to speak to me as you might to a young child or a golden retriever."
January 5, 2025 at 4:16 AM
Hey Quant Folks: Does anyone have any recommendations for testing if two correlation matrices are meaningfully and/or statistically significantly different from each other?
NB: In the words of Jeremy Irons: "Please feel free to speak to me as you might to a young child or a golden retriever."
NB: In the words of Jeremy Irons: "Please feel free to speak to me as you might to a young child or a golden retriever."
Bluesky has finally reached the maturity that we get the important content
If anyone wants to know what half a cup of coffee spilled into a breakfast plate looks like it’s this.
Still tasted pretty good though.
Still tasted pretty good though.
November 17, 2024 at 2:14 AM
Bluesky has finally reached the maturity that we get the important content
Reposted by Rick Carlsson
Just thinking, my name sounds Italian enough that I could probably invent a new pasta shape and name it after myself and no one would find it strange.
November 17, 2024 at 1:28 AM
Just thinking, my name sounds Italian enough that I could probably invent a new pasta shape and name it after myself and no one would find it strange.
Reposted by Rick Carlsson
I understand that there was considerable resistance to the introduction of Risk of Bias tools, but I don't know where to find this discussion, which took place before my entry into the world of research. Any suggestions?
November 15, 2024 at 4:37 PM
I understand that there was considerable resistance to the introduction of Risk of Bias tools, but I don't know where to find this discussion, which took place before my entry into the world of research. Any suggestions?
Reposted by Rick Carlsson
And just like that-academic twitter reformed into academic BlueSky. I’ve missed seeing all this interaction and sharing of science.
November 15, 2024 at 12:35 PM
And just like that-academic twitter reformed into academic BlueSky. I’ve missed seeing all this interaction and sharing of science.
Reposted by Rick Carlsson
Gladwell's latest book includes the below conclusion about who we might target during pandemics (from: www.theguardian.com/books/2024/s...).
But the below plot seems to be the main evidence driving this claim, and it's... not very convincing (from: www.pnas.org/doi/epub/10.... )
But the below plot seems to be the main evidence driving this claim, and it's... not very convincing (from: www.pnas.org/doi/epub/10.... )
October 4, 2024 at 9:37 AM
Gladwell's latest book includes the below conclusion about who we might target during pandemics (from: www.theguardian.com/books/2024/s...).
But the below plot seems to be the main evidence driving this claim, and it's... not very convincing (from: www.pnas.org/doi/epub/10.... )
But the below plot seems to be the main evidence driving this claim, and it's... not very convincing (from: www.pnas.org/doi/epub/10.... )
Join our conference on Open Science and Reproducibility. Keynote by Lakens and @annemscheel.bsky.social , with @nulliusinverba.bsky.social live on stage at the record shop / “folkölscafé” (bar) Mono.
opensciencesweden.org/aktiviteter....
opensciencesweden.org/aktiviteter....
May 22, 2024 at 8:14 AM
Join our conference on Open Science and Reproducibility. Keynote by Lakens and @annemscheel.bsky.social , with @nulliusinverba.bsky.social live on stage at the record shop / “folkölscafé” (bar) Mono.
opensciencesweden.org/aktiviteter....
opensciencesweden.org/aktiviteter....
Reposted by Rick Carlsson
Call for commentaries on “Replication value as a function of citation impact and sample size”.
We ask the members of the scientific community to provide substantive criticism, interpretation, or elaboration on the article by submitting commentaries to Meta-Psychology. open.lnu.se/index.php/me...
We ask the members of the scientific community to provide substantive criticism, interpretation, or elaboration on the article by submitting commentaries to Meta-Psychology. open.lnu.se/index.php/me...
LnuOpen
| Meta-Psychology
open.lnu.se
March 17, 2024 at 8:25 PM
Call for commentaries on “Replication value as a function of citation impact and sample size”.
We ask the members of the scientific community to provide substantive criticism, interpretation, or elaboration on the article by submitting commentaries to Meta-Psychology. open.lnu.se/index.php/me...
We ask the members of the scientific community to provide substantive criticism, interpretation, or elaboration on the article by submitting commentaries to Meta-Psychology. open.lnu.se/index.php/me...
Reposted by Rick Carlsson
Interested in replication research? Consider writing a commentary for the next special issue in Meta-Psychology which will publish critiques of the article "Replication value as a function of citation impact and sample size": open.lnu.se/index.php/me...
March 18, 2024 at 11:22 AM
Interested in replication research? Consider writing a commentary for the next special issue in Meta-Psychology which will publish critiques of the article "Replication value as a function of citation impact and sample size": open.lnu.se/index.php/me...