Magnus Johansson
@pgmj.bsky.social
PhD & lic. psychologist. Research specialist at Karolinska Institutet. R package for Rasch psychometrics: pgmj.github.io/easyRasch/
#openscience, #prevention, #psychometrics, #rstats, #photo
Psychology
41%
Sociology
12%
Pinned
Magnus Johansson
@pgmj.bsky.social
· Mar 11
Detecting item misfit in Rasch models
pgmj.github.io
My simulation study on item misfit detection in Rasch models is published. We should leave rule-of-thumb critical values for GOF metrics behind us and use simulation/bootstrap methods to determine cutoffs appropriate for the data and items being analyzed. pgmj.github.io/rasch_itemfit/ #psychometrics
Maybe try altdoc.etiennebacher.com
Reposted by Magnus Johansson
Two-group pre/post data are deceptively simple, and you could analyze them in many different ways, depending on your goals. Here are three blog posts on the topic:
solomonkurz.netlify.app/blog/2022-06...
solomonkurz.netlify.app/blog/2020-12...
solomonkurz.netlify.app/blog/2023-06...
solomonkurz.netlify.app/blog/2022-06...
solomonkurz.netlify.app/blog/2020-12...
solomonkurz.netlify.app/blog/2023-06...
Sage Journals: Discover world-class research
Subscription and open access journals from Sage, the world's leading independent academic publisher.
journals.sagepub.com
I think the impact is pretty clearly disastrous. But can't blame the paper for all of it's misuse. See for instance:
journals.sagepub.com/doi/full/10....
journals.sagepub.com/doi/full/10....
Reposted by Magnus Johansson
I wish we could ignore Bill Gates on the climate crisis. But he’s a billionaire, so we can’t | George Monbiot
Money talks – and his essay denouncing ‘near-term emissions goals’ at Cop30 mostly argues the case for letting the ultra-rich off the hook
www.theguardian.com
“…the net worth of the 10 richest US billionaires grew by $698bn in the past year. That money alone, the increment in the wealth of 10 people, is almost 10 times the annual amount required to end extreme poverty worldwide.”
www.theguardian.com/commentisfre...
www.theguardian.com/commentisfre...
www.reuters.com
Meta earns $3.5 billion every six months from showing Faceboon and Instagram users 15 billion “higher legal risk” scam ad impressions a day, internal documents state.
That haul vastly exceeds how much the company expects regulators
To fine it for running scam ads.
www.reuters.com/investigatio...
That haul vastly exceeds how much the company expects regulators
To fine it for running scam ads.
www.reuters.com/investigatio...
Waiting for a new work Mac, I’m using an M1 Air 2020 which only has 4 high performance cores. Running some tests in R, the Air is twice as fast when using only 4 cores compared to using 8 cores (including the high efficiency cores).
A recent paper using the 4-item perceived stress scale is citing our PSS #psychometrics paper in the methods section. From our abstract: “the PSS-4 was not deemed suitable as a unidimensional scale”. They didn’t even read the abstract?! doi.org/10.1186/s128...
journals.sagepub.com/doi/full/10....
journals.sagepub.com/doi/full/10....
I was just notified of another paper citing our paper as validating the Swedish PSS-4, when we actually conclude that it should not be used. What was even more surprising was that one of my coauthors on the psychometrics paper was also a coauthor on the paper. Not sure what to do with this.
a cartoon of a man in a suit and tie standing in front of a fire escape
ALT: a cartoon of a man in a suit and tie standing in front of a fire escape
media.tenor.com
Forced to deal with Microsoft Outlook/etc at work, I constantly find the default settings to be annoying/stupid. I wonder if they ever gather data to understand user preferences or just make assumptions. Or am I the outlier?
Here’s How the AI Crash Happens
The U.S. is becoming an Nvidia-state.
www.theatlantic.com
so @matteowong.bsky.social & I wrote on data centers: arguably the most important buildings in the world & are, in a way, holding the economy hostage. Byzantine financial instruments, private equity, depreciating tech, hype, $trillion valuations. it’s all there. an ai crash prob starts here.
Reposted by Richard McElreath, Magnus Johansson, Barrett , and 1 more Richard McElreath, Magnus Johansson, Barrett, Matti Vuorre
I'm looking for a doctoral student with Bayesian background to work on Bayesian workflow and cross-validation (see my publication list users.aalto.fi/~ave/publica... for my recent work) at Aalto University.
Apply through the ELLIS PhD program (dl October 31) ellis.eu/news/ellis-p...
Apply through the ELLIS PhD program (dl October 31) ellis.eu/news/ellis-p...
ELLIS PhD Program: Call for Applications 2025
The ELLIS mission is to create a diverse European network that promotes research excellence and advances breakthroughs in AI, as well as a pan-European PhD program to educate the next generation of AI...
ellis.eu
I'm now also looking for a postdoc with strong Bayesian background and interest in developing Bayesian cross-validation theory, methods and software. Apply by email with no specific deadline (see contact information at users.aalto.fi/~ave/).
Others, please share
Others, please share
🎉 @rpsychologist.com 's PowerLMM.js is the online statistics application of the year 2025 🎉
powerlmmjs.rpsychologist.com
- Calculate power (etc) for multilevel models
- Examine effects of dropout and other important parameters
- Fast! (Instant results)
powerlmmjs.rpsychologist.com
- Calculate power (etc) for multilevel models
- Examine effects of dropout and other important parameters
- Fast! (Instant results)
New release of PowerLMM.js! Browser-based power analysis for longitudinal models with dropout.
Now includes:
- Power analysis summary report
- Reproducible & shareable configs (URL/JSON)
- Calculations validated against R
- Hypothesis region visualization
powerlmmjs.rpsychologist.com
Now includes:
- Power analysis summary report
- Reproducible & shareable configs (URL/JSON)
- Calculations validated against R
- Hypothesis region visualization
powerlmmjs.rpsychologist.com
in my post-academic era i have become extremely scathing of the way academia considers papers to the primary outcome of intellectual work, and refuses to give credit to academics who write software or build other tooling. in industry we very rarely read your papers; but we always use your software
We wrote an article explaining why you shouldn't put several variables into a regression model and report which are statistically significant - even as exploratory research. bmjmedicine.bmj.com/content/4/1/.... How did we do?
I've done some more work on the relative measurement uncertainty, comparing `brms` posterior draws to "plausible values" in Rasch models, and some other reliability metrics. Estimating RMU from draws adds some variation, as shown in the figure.
pgmj.github.io/reliability....
#rstats #psychometrics
pgmj.github.io/reliability....
#rstats #psychometrics
Hmm. I thought I was getting these prompts since I use an always on VPN.
Reposted by Magnus Johansson
Anthony Biglan's "The Nurture Effect" shines a light on how we can intentionally design better societies. It's about building upstream prevention and fostering nurturing environments. What are your thoughts? snapt.io/CUHnJ #SocialScience #CommunityDevelopment #Prevention Image: Vectzeey
Reposted by Diane Coyle, Anna O. Law, Magnus Johansson , and 3 more Diane Coyle, Anna O. Law, Magnus Johansson, Smith, John Hogan, Mark Rice
Carl Zimmer
@carlzimmer.com
· 18d
Largest study of its kind shows AI assistants misrepresent news content 45% of the time – regardless of language or territory
An intensive international study was coordinated by the European Broadcasting Union (EBU) and led by the BBC
www.bbc.co.uk
Largest study of its kind shows AI assistants misrepresent news content 45% of the time – regardless of language or territory. www.bbc.co.uk/mediacentre/...
Yes, there is a vignette here: rpubs.com/dmcneish/102...
Nice! It would be great to see dynamic/adaptive model fit cutoffs mentioned and a stronger recommendation to avoid Hu & Bentler and other rule-of-thumb cutoffs. See for instance pubmed.ncbi.nlm.nih.gov/36787513/ & doi.org/10.1177/0013...
Against Publishing: universonline.nl/nieuws/2025/...
Preprints are read, shared, and cited, yet still dismissed as incomplete until blessed by a publisher. I argue that the true measure of scholarship lies in open exchange, not in the industry’s gatekeeping of what counts as published.
Preprints are read, shared, and cited, yet still dismissed as incomplete until blessed by a publisher. I argue that the true measure of scholarship lies in open exchange, not in the industry’s gatekeeping of what counts as published.
Very useful extension! I used an earlier version in this paper: pgmj.github.io/rasch_itemfit/
Wonderful, thanks!
Do you have one (or more) key references for that? Would be nice to have handy as ppl keep being curious about these models
Magnus Johansson
@pgmj.bsky.social
· Oct 11
Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? - Behavior Research Methods
Scale scores in psychology studies are commonly accompanied by a reliability coefficient like alpha. Coefficient alpha is an index that summarizes reliability across the entire score distribution, implying equal precision for all scores. However, an underappreciated fact is that reliability can be conditional such that scores in certain parts of the score distribution may be more reliable than others. This conditional perspective of reliability is common in item response theory (IRT), but psychologists are generally not well versed in IRT. Correspondingly, the representativeness of a single summary index like alpha across the entire score distribution can be unclear but is rarely considered. If conditional reliability is fairly homogeneous across the score distribution, coefficient alpha may be sufficiently representative and a useful summary. But, if conditional reliability is heterogeneous across the score distribution, alpha may be unrepresentative and may not align with the reliability of a typical score in the data or with a particularly important score like a cut point where decisions are made. This paper proposes a method, R package, and Shiny application to quantify the potential differences between coefficient alpha and conditional reliability across the score distribution. The goal is to facilitate comparisons between conditional reliability and reliability summary indices so that psychologists can contextualize the reliability of their scores more clearly and comprehensively.
doi.org
Re-updated the page and the R package, including the conditional reliability method by McNeish & Dumas as well.
McNeish & Dumas (2025). Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
McNeish & Dumas (2025). Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
Magnus Johansson
@pgmj.bsky.social
· Oct 11
This is what I find awesome (in the truest sense) about scientific development in the modern age. One group posts a new technique and within a few days it is implemented by others. #rstats is magic (and efficient)
This is really neat. I have borrowed the reliability() function to my `easyRasch` package, and use plausible values instead of fully Bayesian estimation to produce similar estimates/CIs, see code example below. RMU point estimates are similar to EAP reliability.
pgmj.github.io/easyRasch/re...
pgmj.github.io/easyRasch/re...
I agree. #openscience combined with #opensource software is great!
Magnus Johansson
@pgmj.bsky.social
· Oct 10
The RelRep.R function "just works" with a dataframe with items as columns and produces pretty neat output for alpha or omega. Point estimate and CI for alpha is almost identical to RMU for my example with `eRm::raschdat1[,1:20]` data. github.com/melissagwolf...
Reposted by Magnus Johansson
This is really neat. I have borrowed the reliability() function to my `easyRasch` package, and use plausible values instead of fully Bayesian estimation to produce similar estimates/CIs, see code example below. RMU point estimates are similar to EAP reliability.
pgmj.github.io/easyRasch/re...
pgmj.github.io/easyRasch/re...
New preprint with @rogierk.bsky.social @paulbuerkner.com - we introduce "relative measurement uncertainty" - a reliability estimation method that's applicable across a broad class of Bayesian measurement models (e.g., generative-, computational- and item response theory-models osf.io/h54k8
This is what I find awesome (in the truest sense) about scientific development in the modern age. One group posts a new technique and within a few days it is implemented by others. #rstats is magic (and efficient)
Magnus Johansson
@pgmj.bsky.social
· Oct 10
Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? - Behavior Research Methods
Scale scores in psychology studies are commonly accompanied by a reliability coefficient like alpha. Coefficient alpha is an index that summarizes reliability across the entire score distribution, implying equal precision for all scores. However, an underappreciated fact is that reliability can be conditional such that scores in certain parts of the score distribution may be more reliable than others. This conditional perspective of reliability is common in item response theory (IRT), but psychologists are generally not well versed in IRT. Correspondingly, the representativeness of a single summary index like alpha across the entire score distribution can be unclear but is rarely considered. If conditional reliability is fairly homogeneous across the score distribution, coefficient alpha may be sufficiently representative and a useful summary. But, if conditional reliability is heterogeneous across the score distribution, alpha may be unrepresentative and may not align with the reliability of a typical score in the data or with a particularly important score like a cut point where decisions are made. This paper proposes a method, R package, and Shiny application to quantify the potential differences between coefficient alpha and conditional reliability across the score distribution. The goal is to facilitate comparisons between conditional reliability and reliability summary indices so that psychologists can contextualize the reliability of their scores more clearly and comprehensively.
doi.org
Looking closer at McNeish & Dumas (2025). How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
The RelRep.R function seems like it could be combined with your approach as a conditional reliability metric. I'll probably have a go at it.
The RelRep.R function seems like it could be combined with your approach as a conditional reliability metric. I'll probably have a go at it.