Matthew B. Jané
@matthewbjane.bsky.social
PhD student at UConn | Analyst at the Medical Evidence Project | Center for Scientific Integrity| matthewbjane.com
Update on Error Report #2: The impact of acoustic stimulation during sleep on memory and sleep architecture: A meta-analysis
The article has been retracted by agreement with the journal editor and authors #metascience
The article has been retracted by agreement with the journal editor and authors #metascience
June 19, 2025 at 9:11 PM
Update on Error Report #2: The impact of acoustic stimulation during sleep on memory and sleep architecture: A meta-analysis
The article has been retracted by agreement with the journal editor and authors #metascience
The article has been retracted by agreement with the journal editor and authors #metascience
Reposted by Matthew B. Jané
"Meta-meta-analyses trade precision & rigor for expediency. If your aim is rigor, there’s no substitute for doing the hard work."
Preach!
I once read a #meta-analysis that mistook SE for SD. Since then I am suspicious of all such studies.
Do the hard work!
matthewbjane.github.io/blog-posts/b...
Preach!
I once read a #meta-analysis that mistook SE for SD. Since then I am suspicious of all such studies.
Do the hard work!
matthewbjane.github.io/blog-posts/b...
When Evidence Synthesis Obscures: A Critique of Meta-Meta-Analyses – Matthew B. Jané
Meta-analyses are often seen as the gold standard of evidence synthesis–systematic, quantitative, and rigorous (at least when done well). So it’s no surprise that some researchers have taken things a ...
matthewbjane.github.io
May 26, 2025 at 5:18 AM
"Meta-meta-analyses trade precision & rigor for expediency. If your aim is rigor, there’s no substitute for doing the hard work."
Preach!
I once read a #meta-analysis that mistook SE for SD. Since then I am suspicious of all such studies.
Do the hard work!
matthewbjane.github.io/blog-posts/b...
Preach!
I once read a #meta-analysis that mistook SE for SD. Since then I am suspicious of all such studies.
Do the hard work!
matthewbjane.github.io/blog-posts/b...
Reposted by Matthew B. Jané
This is very nice from @matthewbjane.bsky.social as to why umbrella reviews are basically useless (sorry!), essentially because they compound up the issues with the meta analyses they include. I have the same concerns but articulate them less eloquently. matthewbjane.github.io/blog-posts/b...
When Evidence Synthesis Obscures: A Critique of Meta-Meta-Analyses – Matthew B. Jané
Meta-analyses are often seen as the gold standard of evidence synthesis–systematic, quantitative, and rigorous (at least when done well). So it’s no surprise that some researchers have taken things a ...
matthewbjane.github.io
May 25, 2025 at 9:24 AM
This is very nice from @matthewbjane.bsky.social as to why umbrella reviews are basically useless (sorry!), essentially because they compound up the issues with the meta analyses they include. I have the same concerns but articulate them less eloquently. matthewbjane.github.io/blog-posts/b...
Reposted by Matthew B. Jané
I've moaned about meta-analyses of meta-analyses for a while and had several interesting experiences reviewing them... Now I can save time and just send people to Matt's post to explain why not to do them.
Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses.
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
May 24, 2025 at 8:40 AM
I've moaned about meta-analyses of meta-analyses for a while and had several interesting experiences reviewing them... Now I can save time and just send people to Matt's post to explain why not to do them.
Reposted by Matthew B. Jané
Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses.
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
May 23, 2025 at 10:47 PM
Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses.
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses.
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
May 23, 2025 at 10:47 PM
Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses.
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Link: matthewbjane.github.io/blog-posts/b...
#stats #metascience
Reposted by Matthew B. Jané
I'm calling for a total and complete shutdown of JAMA Pediatrics until we can figure out what is going on goodscience.substack.com/p/the-resear... I mean, come ON
April 29, 2025 at 12:45 PM
I'm calling for a total and complete shutdown of JAMA Pediatrics until we can figure out what is going on goodscience.substack.com/p/the-resear... I mean, come ON
(#metascience) A good article by @stuartbuck.bsky.social on the Fluoride and IQ meta-analysis: goodscience.substack.com/p/the-resear.... Summarizes some of the points in our forensic review of it (see the forensic review as a pre-print here: osf.io/preprints/os...). Very nice excerpt below:
April 28, 2025 at 5:01 PM
(#metascience) A good article by @stuartbuck.bsky.social on the Fluoride and IQ meta-analysis: goodscience.substack.com/p/the-resear.... Summarizes some of the points in our forensic review of it (see the forensic review as a pre-print here: osf.io/preprints/os...). Very nice excerpt below:
Reposted by Matthew B. Jané
When a STALT error messes up your pagination.
April 24, 2025 at 1:52 AM
When a STALT error messes up your pagination.
Reposted by Matthew B. Jané
Error Report #4: matthewbjane.quarto.pub/meta-analysi...
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
April 23, 2025 at 9:07 PM
Error Report #4: matthewbjane.quarto.pub/meta-analysi...
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
Error Report #4: matthewbjane.quarto.pub/meta-analysi...
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
April 23, 2025 at 9:07 PM
Error Report #4: matthewbjane.quarto.pub/meta-analysi...
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os...
#metascience
Reposted by Matthew B. Jané
Spent the past couple days looking at meta-analyses in body comp literature to get reasonable parameter estimates to use in simulations for a study we're planning and really trying to avoid giving myself a to-do list of meta-analyses that I need to correct/redo 🙃 cc: @matthewbjane.bsky.social
April 17, 2025 at 8:01 AM
Spent the past couple days looking at meta-analyses in body comp literature to get reasonable parameter estimates to use in simulations for a study we're planning and really trying to avoid giving myself a to-do list of meta-analyses that I need to correct/redo 🙃 cc: @matthewbjane.bsky.social
Reposted by Matthew B. Jané
livefreeordichotomize.com by @lucystats.bsky.social
matthewbjane.github.io/blog.html by @matthewbjane.bsky.social
www.andrewheiss.com/blog/ by the king @andrew.heiss.phd
ngreifer.github.io/blog/ by yours truly (can't resist a self-promotion 😉)
Stats blogs are the best!
matthewbjane.github.io/blog.html by @matthewbjane.bsky.social
www.andrewheiss.com/blog/ by the king @andrew.heiss.phd
ngreifer.github.io/blog/ by yours truly (can't resist a self-promotion 😉)
Stats blogs are the best!
Posts | Noah Greifer
Statistical Consultant and Programmer
ngreifer.github.io
April 22, 2025 at 6:26 PM
livefreeordichotomize.com by @lucystats.bsky.social
matthewbjane.github.io/blog.html by @matthewbjane.bsky.social
www.andrewheiss.com/blog/ by the king @andrew.heiss.phd
ngreifer.github.io/blog/ by yours truly (can't resist a self-promotion 😉)
Stats blogs are the best!
matthewbjane.github.io/blog.html by @matthewbjane.bsky.social
www.andrewheiss.com/blog/ by the king @andrew.heiss.phd
ngreifer.github.io/blog/ by yours truly (can't resist a self-promotion 😉)
Stats blogs are the best!
Coding RCTs for a meta-analysis…
April 11, 2025 at 5:18 PM
Coding RCTs for a meta-analysis…
Reposted by Matthew B. Jané
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
April 8, 2025 at 4:58 PM
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
Reposted by Matthew B. Jané
Another great meta-science contribution by @matthewbjane.bsky.social and others.
This and his re-analysis of the “social media causes harm” meta-analysis are great examples of how we can use science to serve society without necessarily having to be “activists”
This and his re-analysis of the “social media causes harm” meta-analysis are great examples of how we can use science to serve society without necessarily having to be “activists”
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
April 9, 2025 at 6:47 AM
Another great meta-science contribution by @matthewbjane.bsky.social and others.
This and his re-analysis of the “social media causes harm” meta-analysis are great examples of how we can use science to serve society without necessarily having to be “activists”
This and his re-analysis of the “social media causes harm” meta-analysis are great examples of how we can use science to serve society without necessarily having to be “activists”
Most studies in the meta-analysis are no more detailed than the following:
“Area A has low fluoride water levels, Area B has high fluoride water levels, and area A has a higher mean IQ”
No matching or balancing of covariates. Distinct geographic areas differ more than just their fluoride levels.
“Area A has low fluoride water levels, Area B has high fluoride water levels, and area A has a higher mean IQ”
No matching or balancing of covariates. Distinct geographic areas differ more than just their fluoride levels.
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
April 8, 2025 at 5:35 PM
Most studies in the meta-analysis are no more detailed than the following:
“Area A has low fluoride water levels, Area B has high fluoride water levels, and area A has a higher mean IQ”
No matching or balancing of covariates. Distinct geographic areas differ more than just their fluoride levels.
“Area A has low fluoride water levels, Area B has high fluoride water levels, and area A has a higher mean IQ”
No matching or balancing of covariates. Distinct geographic areas differ more than just their fluoride levels.
They chose to use the standardized mean difference in IQ between a reference and exposure group however the reference in many studies has more fluoride exposure than the exposure group in other studies! There is no consistent definition for the groupings.
April 8, 2025 at 5:23 PM
They chose to use the standardized mean difference in IQ between a reference and exposure group however the reference in many studies has more fluoride exposure than the exposure group in other studies! There is no consistent definition for the groupings.
One major finding in our pre-print is that the journal ‘Fluoride’ publishes much larger effects than any other academic journal. Fluoride is published by the International Society for Fluoride Research which is run by anti-fluoride activists and hosts water defluoridation workshops. #metascience
April 8, 2025 at 5:16 PM
One major finding in our pre-print is that the journal ‘Fluoride’ publishes much larger effects than any other academic journal. Fluoride is published by the International Society for Fluoride Research which is run by anti-fluoride activists and hosts water defluoridation workshops. #metascience
I saw @epiellie.bsky.social comment on this so I am tagging her here #metascience
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
April 8, 2025 at 5:03 PM
I saw @epiellie.bsky.social comment on this so I am tagging her here #metascience
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
April 8, 2025 at 4:58 PM
The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...
In probability the following is true by definition
Pr(sample space) = 1
Pr(empty set) = 0
Pr(sample space) = 1
Pr(empty set) = 0
Forgot the whole post where it gets strange. Basically he goes from here to saying zero and one aren’t probabilities
www.lesswrong.com/posts/QGkYCw...
www.lesswrong.com/posts/QGkYCw...
0 And 1 Are Not Probabilities — LessWrong
One, two, and three are all integers, and so is negative four. If you keep counting up, or keep counting down, you’re bound to encounter a whole lot…
www.lesswrong.com
April 1, 2025 at 11:17 PM
In probability the following is true by definition
Pr(sample space) = 1
Pr(empty set) = 0
Pr(sample space) = 1
Pr(empty set) = 0
👀
Abstaining from social media does not significantly affect positive affect, negative affect, or life satisfaction, finds
@lauralemahieu.bsky.social et al., in a meta analysis doi.org/10.1038/s415...
@lauralemahieu.bsky.social et al., in a meta analysis doi.org/10.1038/s415...
![Image description
Figure 1. Forest plot showing results from a meta-analysis of studies on digital interventions and their effect on positive affect. Each row lists a study with its estimated effect size (Hedges' g), 95% confidence interval (CI), and weight in the meta-analysis. Most individual study estimates are close to zero, with some negative and some positive. The overall effect size is 0.03 [95% CI: -0.11, 0.16], suggesting a small, non-significant positive effect. The prediction interval spans from -0.42 to 0.47. The plot indicates moderate heterogeneity (I² = 61%, p < .01). A diamond represents the overall effect at the bottom of the plot.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:tpjypws5r3t3xvjqswqnxnf6/bafkreihwbnhwad5vkvfw55lndschavn7fjihyjdm5gpy42k42oakqnpsry@jpeg)
March 26, 2025 at 5:00 PM
👀
Reposted by Matthew B. Jané
We just published a paper on the performance of various tests and confidence intervals for the scale coefficients in location-scale models in meta-analysis: doi.org/10.3758/s134... Overall, REML-based LRTs and profile-likelihood CIs performed best, but the number of studies typically must be large.
Performance of location-scale models in meta-analysis: A simulation study - Behavior Research Methods
Location-scale models in the field of meta-analysis allow researchers to simultaneously study the influence of moderator variables on the mean (location) and variance (scale) of the distribution of tr...
doi.org
March 19, 2025 at 10:47 AM
We just published a paper on the performance of various tests and confidence intervals for the scale coefficients in location-scale models in meta-analysis: doi.org/10.3758/s134... Overall, REML-based LRTs and profile-likelihood CIs performed best, but the number of studies typically must be large.