Ian Hussey | Lightnews

Light up
your news

About Privacy Terms Help

Ian Hussey

Ian Hussey

@ianhussey.mmmdata.io

3.3K followers 890 following 930 posts

Meta-scientist and psychologist. Senior lecturer @unibe.ch‬. Chief recommender @error.reviews. "Jumped up punk who hasn't earned his stripes." All views a product of my learning history. If behaviorism did not exist, it would be necessary to invent it. .. more

Psychology 66%

Sociology 10%

more

Posts Replies Media Videos

Pinned

Ian Hussey @ianhussey.mmmdata.io · Jul 22

Lego Science is research driven by modular convenience.

When researchers combine methods or concepts, more out of convenience than any deep curiosity in the resulting research question, to create publishable units.

"What role does {my favourite construct} play in {task}?"

Reposted by Dorothy Bishop, Ian Hussey

Jamie Cummins @jamiecummins.bsky.social · 23h

abundanceandgrowthblog.substack.com/p/the-case-f...

The case for sharing clinical trial data

The story behind the first statin and how its development was almost derailed, and the implications of sharing clinical trial data.

abundanceandgrowthblog.substack.com

Ian Hussey @ianhussey.mmmdata.io · 1d

Yeah I could hear that. The former can't be answered until to address the latter.

Ian Hussey @ianhussey.mmmdata.io · 1d

Guarding against the highly optimistic hope that heterogeneity represents important moderators hiding in plain sight is one way to do this.

Ian Hussey @ianhussey.mmmdata.io · 1d

Psych spent lots of time in the 2010s chasing its tail about hidden moderators. The concept is, at its core, a challenge to the falsifiability of claims. I think it's important that we don't walk down this cul de sac again.

Ian Hussey @ianhussey.mmmdata.io · 1d

Many of these context effects are interesting - but to test them properly we would need high quality original experiments that directly manipulate them, rather than to rely on evidence from low power, post hoc, non-randomised differences between published studies.

Ian Hussey @ianhussey.mmmdata.io · 1d

The next issue is power. We've known for 25+ years that the statistical power of moderator meta-analysis is usually very low. People often run many moderator tests and don't correct for multiple testing, collectively making for many false positives and false negatives.

doi.org/10.1037/1082...

Ian Hussey @ianhussey.mmmdata.io · 1d

But heterogeneity isn't gold to be mined for information. When, for example, all studies in a meta analysis are rated as high risk of bias (as in Linden et al. 2022), the primary need is to raise the quality of the original research, not to go noise mining in the heterogeneity.

Ian Hussey @ianhussey.mmmdata.io · 1d

I appreciate the desire to pursue Paul (1967) question of "What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?"

Ian Hussey @ianhussey.mmmdata.io · 1d

Second, lots of variation isn't meaningful, and we need to continually cautious about not over interpreting it. Mere methodological variation between studies accounts for lots of variation - even according to this author's own work elsewhere.

Sage Journals: Discover world-class research

Subscription and open access journals from Sage, the world's leading independent academic publisher.

journals.sagepub.com

Ian Hussey @ianhussey.mmmdata.io · 1d

RE models assume effect sizes are drawn from a normal distribution centered on the mean effect. There's no single "true" effect but a distribution of true effects with quantified variation (tau) that is not noise. Heterogeneity is already a feature of the model itself.

Ian Hussey @ianhussey.mmmdata.io · 1d

First, we don't need to we should drop the assumption that "interventions will have a single, underlying effect size", because that's not how Random Effects meta-analysis models work.

Ian Hussey @ianhussey.mmmdata.io · 1d

I think some of these points miss the mark. Moreso, I'm concerned that they'll be used by others to smuggle hidden moderators debates back into psychology.

🧵

www.bps.org.uk/psychologist...

Psychology needs a… heterogeneity revolution | BPS

Audrey Linden argues we should drop the assumption that interventions will have a single, underlying effect size.

Reposted by Ian Hussey

James Wade @jameshwade.bsky.social · 1d

I built an R package that turns Shiny apps into UIs that render directly inside Claude Desktop or ChatGPT.

It's called shinymcp. Drop-downs, plots, tables all inline in the chat.

github.com/jameshwade/shinymcp

Reposted by Ian Hussey

Randy Ellis @randalljellis.bsky.social · 1d

Here's my conversation with Mu Yang on Metascience Matters: www.youtube.com/watch?v=E2EK...

We discussed her work as a scientific sleuth, academic incentives for positive data, individual cases she has pursued, and why she loves being a sleuth.

Also on Spotify: open.spotify.com/episode/16R6...

300+ retractions, image manipulation, and why science should be boring | Metascience Matters #3

YouTube video by Metascience Matters

www.youtube.com

Ian Hussey @ianhussey.mmmdata.io · 5d

{psychdsish} R package to create standardized structured projects for psychological research projects processed and analyzed in R:

github.com/ianhussey/ps...

Reposted by James E. Bartlett

Ian Hussey @ianhussey.mmmdata.io · 5d

Early draft of my ebook for the course:

ianhussey.quarto.pub/reproducible...

Ian Hussey @ianhussey.mmmdata.io · 5d

Basic questions like "how many participants are in the dataset?" can produce surprisingly different answers between analysts.

Results from students in my class in data wrangling in tidyverse, who are good at wrangling but still have to make semi-subjective choices:

Reposted by Ian Hussey, Paul Goldsmith-Pinkham, Barrett

Davis Vaughan @davisvaughan.bsky.social · 6d

dplyr 1.2.0 is out now and we are SO excited!

- `filter_out()` for dropping rows

- `recode_values()`, `replace_values()`, and `replace_when()` that join `case_when()` as a complete family of recoding/replacing tools

These are huge quality of life wins for #rstats!

tidyverse.org/blog/2026/02...

dplyr 1.2.0 fills in some important gaps in dplyr's API: we've added a new complement to `filter()` focused on dropping rows, and we've expanded the `case_when()` family with three new recoding and re...

Reposted by Scott L. Greer, Calvin K. Lai, Cameron Brick , and 12 more Scott L. Greer, Calvin K. Lai, Cameron Brick, Ian Hussey, Max Nathan, Lauren Hersch Nicholas, Nazita Lajevardi, Margot C. Finn, Sofia Stathi, Patrick Präg, Marko Lindroos, Adele Perry, Simon Wiederhold, Claudia Landwehr, Richard Moorhead

John Holbein @johnholbein1.bsky.social · 6d

Are female economists treated differently than males in academic seminars?

These authors wanted to know whether gender shapes how scholars are treated when presenting research.

So they built a massive dataset of 2,000+ economics seminars, job talks, and conference presentations from 2019–2023...

Ian Hussey @ianhussey.mmmdata.io · 6d

"We, the arbiters of correction notices, retractions, and future publications - the major reinforcers for academics - have no levers to pull on to enforce this" 👀

If failure to supply data upon request after committing to do so resulted in a correction notice, researchers would comply immediately.

Reposted by Ingo Rohlfing, Ian Hussey, José Pina-Sánchez

Darren Dahly @statsepi.bsky.social · 6d

Don't you f**king dare.

Screen shot of a youtube video titled "AI.FILL Function Explained: 10X Productivity in Excel with AI" with the caption "Let ChatGPT fill your missing data"

Ian Hussey @ianhussey.mmmdata.io · 7d

If you're interested in data sleuthing but aren't sure where to start,

or if you're conducting a systematic review/meta-analysis and want to ensure you're not including junk studies,

check out this Cochrane training session on Trustworthiness Assessment by @jdwilko.bsky.social

INSPECT-SR: A tool for assessing trustworthiness of randomised controlled trials | Cochrane

www.cochrane.org

Reposted by Ian Hussey

Nick DeVito @ndevito1.bsky.social · 7d

My own personal example of this for a study I got rejected in one place as a reviewer and then got published at another, unchanged.

Look at the two citations in the image. The years alone should give away they don't support the claim made in the paper.

www.sciencedirect.com/science/arti...

Ian Hussey @ianhussey.mmmdata.io · 8d

This is the difference between "the literature is a mess" in the sense of being able to integrate findings from different papers without fine grain additional info vs "individual papers in the literature are a mess". The analyses we provide show convincingly that the literature is a mess.

Reposted by Ian Hussey, Patrick S. Forscher, Jesper Wulff

Richard McElreath 🐈‍⬛ @rmcelreath.bsky.social · 8d

Wow this scoring chaos seems to be an extreme case of what I say about many ad hoc analyses: no derivation of method from a clear scientific theory, no assessment of statistical properties, and decades pass before someone notices. This happens in biology too, so let’s not pick on psychology only

Tom Stafford @tomstafford.mastodon.online.ap.brid.gy · 8d

Read my latest post for reflections on reproducibility, research quality and a summary of a great new study which shows how NOT to do it

https://open.substack.com/pub/tomstafford/p/gambling-with-research-quality

Gambling with research quality

How you get 244 different ways to measure performance on the same test of decision making. And what it means for the reliability of behavioural science

tomstafford.substack.com

Reposted by Ian Hussey

Ioana A. Cristea @ioanaacristea.bsky.social · 8d

But data is available from the authors upon reasonable request. For a meta-research paper this is simply ridiculous
Honestly, read our paper more

pubmed.ncbi.nlm.nih.gov/35623542/

Ian Hussey @ianhussey.mmmdata.io · 8d

8.0% of citations in the medical literature contain major errors.

E.g., the cited work makes the opposite claim or is unrelated to the claim in the citing article.

Systematic review and meta-analysis of quotation inaccuracy in medicine - Research Integrity and Peer Review

Background Quotations are crucial to science but have been shown to be often inaccurate. Quotation errors, that is, a reference not supporting the authors’ claim, may still be a significant issue in s...

link.springer.com

Ian Hussey @ianhussey.mmmdata.io · 8d

@anniria.bsky.social second attempt at tagging you

Ian Hussey @ianhussey.mmmdata.io · 8d

"you could - by careful choice of an existing scoring method from the literature - find any effect, or nothing, or the reverse of any effect you choose. This is bonkers. ... so extreme as to be farcical."

This is the sentiment we were hoping people would come away with!

w/@anniria.bsky.social

Tom Stafford @tomstafford.mastodon.online.ap.brid.gy · 8d

Read my latest post for reflections on reproducibility, research quality and a summary of a great new study which shows how NOT to do it

https://open.substack.com/pub/tomstafford/p/gambling-with-research-quality

Gambling with research quality

How you get 244 different ways to measure performance on the same test of decision making. And what it means for the reliability of behavioural science

tomstafford.substack.com

Ian Hussey @ianhussey.mmmdata.io · 8d

Separately, the COMPare project spent a lot of time tracking outcome switching and reporting it to journals, who usually did nothing about it. It was labor intensive, but @jamiecummins.bsky.social is working with them to create a semi-automated version of it as a branch of regcheck.