Isaac Handley-Miner
@ihandleyminer.bsky.social
Psychology postdoc studying how people think about truth and credibility, and interested in open science and metascience.
“We are hungry for the curtain to be pulled back … we want all the gory details and imperfections of a study out on the table, open and transparent for deep consideration. It is with and through this transparency that we step from the world of less good practices to the world of better ones.”
My time as Editor of AMPPS is coming to an end-- here are some parting thoughts. journals.sagepub.com/doi/full/10.... .
Taking Stock of Advances in Methods and Practices in Psychological Science at the End of the Beginning - David A. Sbarra, 2025
journals.sagepub.com
November 27, 2025 at 12:56 PM
“We are hungry for the curtain to be pulled back … we want all the gory details and imperfections of a study out on the table, open and transparent for deep consideration. It is with and through this transparency that we step from the world of less good practices to the world of better ones.”
Reposted by Isaac Handley-Miner
Interesting new special issue in Psychological Assessment.
Edited by Kristin Naragon-Gainey and @kstanton.bsky.social
Here's their overview paper: psycnet.apa.org/record/2026-...
And here's our contribution, which will win us no friends:
psycnet.apa.org/record/2026-...
Edited by Kristin Naragon-Gainey and @kstanton.bsky.social
Here's their overview paper: psycnet.apa.org/record/2026-...
And here's our contribution, which will win us no friends:
psycnet.apa.org/record/2026-...
October 24, 2025 at 4:00 PM
Interesting new special issue in Psychological Assessment.
Edited by Kristin Naragon-Gainey and @kstanton.bsky.social
Here's their overview paper: psycnet.apa.org/record/2026-...
And here's our contribution, which will win us no friends:
psycnet.apa.org/record/2026-...
Edited by Kristin Naragon-Gainey and @kstanton.bsky.social
Here's their overview paper: psycnet.apa.org/record/2026-...
And here's our contribution, which will win us no friends:
psycnet.apa.org/record/2026-...
Reposted by Isaac Handley-Miner
ANSWER: 0 (yes, ZERO!)
This is a result of an analysis done by a student in my grad seminar, using a large dataset (N=307,313).
What this result might mean: Nobody's personality is truly "average," and people's personality profiles (at least Big 5) might be more "jagged" than we think.
(🧵 1/5)
This is a result of an analysis done by a student in my grad seminar, using a large dataset (N=307,313).
What this result might mean: Nobody's personality is truly "average," and people's personality profiles (at least Big 5) might be more "jagged" than we think.
(🧵 1/5)
Imagine you have Big 5 personality scores from over 300,000 people. You designate the scores in the "mean +/- 0.25 SDs" range for each trait (~20%) as the average range.
QUESTION: How many people in this >300K sample do you think fall in the average range for ALL 5 TRAITS?
What's your answer?
QUESTION: How many people in this >300K sample do you think fall in the average range for ALL 5 TRAITS?
What's your answer?
September 7, 2025 at 3:03 PM
ANSWER: 0 (yes, ZERO!)
This is a result of an analysis done by a student in my grad seminar, using a large dataset (N=307,313).
What this result might mean: Nobody's personality is truly "average," and people's personality profiles (at least Big 5) might be more "jagged" than we think.
(🧵 1/5)
This is a result of an analysis done by a student in my grad seminar, using a large dataset (N=307,313).
What this result might mean: Nobody's personality is truly "average," and people's personality profiles (at least Big 5) might be more "jagged" than we think.
(🧵 1/5)
Reposted by Isaac Handley-Miner
v1.108 is rolling out today 🚚
Now live, at long last: Bookmarks, aka Saved Posts. For all those posts you'll definitely plan to come back to!
Update the app and give it a try. The button is right down there 👇
Now live, at long last: Bookmarks, aka Saved Posts. For all those posts you'll definitely plan to come back to!
Update the app and give it a try. The button is right down there 👇
September 8, 2025 at 6:24 PM
v1.108 is rolling out today 🚚
Now live, at long last: Bookmarks, aka Saved Posts. For all those posts you'll definitely plan to come back to!
Update the app and give it a try. The button is right down there 👇
Now live, at long last: Bookmarks, aka Saved Posts. For all those posts you'll definitely plan to come back to!
Update the app and give it a try. The button is right down there 👇
Reposted by Isaac Handley-Miner
As part of the Pilot Reporting Task Force, founded at @improvingpsych.org conference 2023, I would like to share a preprint with you: osf.io/preprints/ps.... We are looking for feedback on our perspective piece, promoting more transparency in the reporting of piloting in psychological science. 1/4
OSF
osf.io
September 3, 2025 at 4:07 PM
As part of the Pilot Reporting Task Force, founded at @improvingpsych.org conference 2023, I would like to share a preprint with you: osf.io/preprints/ps.... We are looking for feedback on our perspective piece, promoting more transparency in the reporting of piloting in psychological science. 1/4
Reposted by Isaac Handley-Miner
"Don't seat the baby near the door" 👶 Thanks to @infantstudies.bsky.social for the chance to discuss our task force work on making pilot study insights more transparent to improve psyc science!
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
New on the Baby Blog - Making pilots public and improving developmental science by @agataboch.bsky.social and @mbneff.bsky.social #EarlyYears #DevPsySky #PsychSciSky #science #research #infantstudies infantstudies.org/making-pilot...
September 3, 2025 at 9:01 AM
"Don't seat the baby near the door" 👶 Thanks to @infantstudies.bsky.social for the chance to discuss our task force work on making pilot study insights more transparent to improve psyc science!
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
Reposted by Isaac Handley-Miner
For those interested, here's our full preprint on the call for more transparent pilot study reporting: osf.io/preprints/ps...
@ihandleyminer.bsky.social #OpenScience #PilotStudies
@ihandleyminer.bsky.social #OpenScience #PilotStudies
"Don't seat the baby near the door" 👶 Thanks to @infantstudies.bsky.social for the chance to discuss our task force work on making pilot study insights more transparent to improve psyc science!
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
@agataboch.bsky.social #OpenScience #DevPsySky #PilotStudies #PsychScience #Transparency
New on the Baby Blog - Making pilots public and improving developmental science by @agataboch.bsky.social and @mbneff.bsky.social #EarlyYears #DevPsySky #PsychSciSky #science #research #infantstudies infantstudies.org/making-pilot...
September 3, 2025 at 9:05 AM
For those interested, here's our full preprint on the call for more transparent pilot study reporting: osf.io/preprints/ps...
@ihandleyminer.bsky.social #OpenScience #PilotStudies
@ihandleyminer.bsky.social #OpenScience #PilotStudies
Reposted by Isaac Handley-Miner
New 📰
Demand characteristics are a fundamental methodological concern in research with humans
Yet little is known about the magnitude and consistency of these effects
In a new paper, I, M. Wyatt, and @mcxfrank.bsky.social take stock of what we’ve learned via meta-analysis
doi.org/10.1525/coll...
Demand characteristics are a fundamental methodological concern in research with humans
Yet little is known about the magnitude and consistency of these effects
In a new paper, I, M. Wyatt, and @mcxfrank.bsky.social take stock of what we’ve learned via meta-analysis
doi.org/10.1525/coll...
A Meta-Analysis of the Impact and Heterogeneity of Explicit Demand Characteristics
Demand characteristics are a fundamental methodological concern in experimental psychology. Yet, little is known about the direction, magnitude, and consistency of their effects. We conducted a three-...
doi.org
August 14, 2025 at 4:39 PM
New 📰
Demand characteristics are a fundamental methodological concern in research with humans
Yet little is known about the magnitude and consistency of these effects
In a new paper, I, M. Wyatt, and @mcxfrank.bsky.social take stock of what we’ve learned via meta-analysis
doi.org/10.1525/coll...
Demand characteristics are a fundamental methodological concern in research with humans
Yet little is known about the magnitude and consistency of these effects
In a new paper, I, M. Wyatt, and @mcxfrank.bsky.social take stock of what we’ve learned via meta-analysis
doi.org/10.1525/coll...
Reposted by Isaac Handley-Miner
Funding opportunity for SIPS members!
The Society for the Improvement of Psychological Science and Peer Community In Registered Reports invite researchers to submit research projects for consideration for research funding from SIPS and a recommendation decision from PCI-RR.
buff.ly/AigqCod
The Society for the Improvement of Psychological Science and Peer Community In Registered Reports invite researchers to submit research projects for consideration for research funding from SIPS and a recommendation decision from PCI-RR.
buff.ly/AigqCod
July 23, 2025 at 7:20 AM
Funding opportunity for SIPS members!
The Society for the Improvement of Psychological Science and Peer Community In Registered Reports invite researchers to submit research projects for consideration for research funding from SIPS and a recommendation decision from PCI-RR.
buff.ly/AigqCod
The Society for the Improvement of Psychological Science and Peer Community In Registered Reports invite researchers to submit research projects for consideration for research funding from SIPS and a recommendation decision from PCI-RR.
buff.ly/AigqCod
Reposted by Isaac Handley-Miner
Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.
July 1, 2025 at 6:26 PM
Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.
Reposted by Isaac Handley-Miner
I'm working on PaperCheck today, and would love to take queries about whether it could do something you'd like to accomplish more easily in your metascience workflow. I'll try to show you how, and maybe even get you to collaborate on adding a module!
scienceverse.github.io/papercheck/
scienceverse.github.io/papercheck/
Check Scientific Papers for Best Practices
A modular, extendable system for automatically checking scientific papers for best practices using text search, R code, and/or LLM queries.
scienceverse.github.io
June 25, 2025 at 10:56 AM
I'm working on PaperCheck today, and would love to take queries about whether it could do something you'd like to accomplish more easily in your metascience workflow. I'll try to show you how, and maybe even get you to collaborate on adding a module!
scienceverse.github.io/papercheck/
scienceverse.github.io/papercheck/
Reposted by Isaac Handley-Miner
Many-analysts approaches can help uncover analytical insecurity. The Journal of Robustness Reports publishes short reports that check alternative analytical approaches for articles. Very cool! 💪 #SIPS2025 @improvingpsych.org #OpenScience
June 25, 2025 at 12:11 PM
Many-analysts approaches can help uncover analytical insecurity. The Journal of Robustness Reports publishes short reports that check alternative analytical approaches for articles. Very cool! 💪 #SIPS2025 @improvingpsych.org #OpenScience
Reposted by Isaac Handley-Miner
We're back, with a fresh new lineup of talks and workshops!
Get your place while you still can to learn about how to make your research more robust and how to deal with scientific fraud!
This event is free and open to all: tinyurl.com/3ewaduvm
Get your place while you still can to learn about how to make your research more robust and how to deal with scientific fraud!
This event is free and open to all: tinyurl.com/3ewaduvm
June 11, 2025 at 9:44 AM
We're back, with a fresh new lineup of talks and workshops!
Get your place while you still can to learn about how to make your research more robust and how to deal with scientific fraud!
This event is free and open to all: tinyurl.com/3ewaduvm
Get your place while you still can to learn about how to make your research more robust and how to deal with scientific fraud!
This event is free and open to all: tinyurl.com/3ewaduvm
Reposted by Isaac Handley-Miner
My editorial on how journals can earn trust.
We often use journal names as proxies for quality. This is bad bc it’s not valid. But it could be. Editors could make journal name a valid signal. And we could place value on journals that show us how they do that.
journals.sagepub.com/doi/full/10....
We often use journal names as proxies for quality. This is bad bc it’s not valid. But it could be. Editors could make journal name a valid signal. And we could place value on journals that show us how they do that.
journals.sagepub.com/doi/full/10....
May 8, 2025 at 9:25 AM
My editorial on how journals can earn trust.
We often use journal names as proxies for quality. This is bad bc it’s not valid. But it could be. Editors could make journal name a valid signal. And we could place value on journals that show us how they do that.
journals.sagepub.com/doi/full/10....
We often use journal names as proxies for quality. This is bad bc it’s not valid. But it could be. Editors could make journal name a valid signal. And we could place value on journals that show us how they do that.
journals.sagepub.com/doi/full/10....
Reposted by Isaac Handley-Miner
What is the future of Open Science?
Join @annemscheel.bsky.social
@nicholascoles.bsky.social @szaszibarnabas.bsky.social @aufdroeseler.bsky.social and
@stephaniemlee.bsky.social for an exciting discussion during the Keynote Panel at the #SIPS2025 Online.
Join @annemscheel.bsky.social
@nicholascoles.bsky.social @szaszibarnabas.bsky.social @aufdroeseler.bsky.social and
@stephaniemlee.bsky.social for an exciting discussion during the Keynote Panel at the #SIPS2025 Online.
April 21, 2025 at 1:09 PM
What is the future of Open Science?
Join @annemscheel.bsky.social
@nicholascoles.bsky.social @szaszibarnabas.bsky.social @aufdroeseler.bsky.social and
@stephaniemlee.bsky.social for an exciting discussion during the Keynote Panel at the #SIPS2025 Online.
Join @annemscheel.bsky.social
@nicholascoles.bsky.social @szaszibarnabas.bsky.social @aufdroeseler.bsky.social and
@stephaniemlee.bsky.social for an exciting discussion during the Keynote Panel at the #SIPS2025 Online.
Reposted by Isaac Handley-Miner
🔬 Want to improve research transparency?
Join this webinar with experts from
@plosbiology.org!
Learn how this format enhances reproducibility & benefits science.
🎤 Dr. @npariente.bsky.social & Dr. Nikki Osborne from
@rrr-research.bsky.social
📅 April 30, 3 PM CEST #OpenScience
Join this webinar with experts from
@plosbiology.org!
Learn how this format enhances reproducibility & benefits science.
🎤 Dr. @npariente.bsky.social & Dr. Nikki Osborne from
@rrr-research.bsky.social
📅 April 30, 3 PM CEST #OpenScience
April 16, 2025 at 10:27 AM
🔬 Want to improve research transparency?
Join this webinar with experts from
@plosbiology.org!
Learn how this format enhances reproducibility & benefits science.
🎤 Dr. @npariente.bsky.social & Dr. Nikki Osborne from
@rrr-research.bsky.social
📅 April 30, 3 PM CEST #OpenScience
Join this webinar with experts from
@plosbiology.org!
Learn how this format enhances reproducibility & benefits science.
🎤 Dr. @npariente.bsky.social & Dr. Nikki Osborne from
@rrr-research.bsky.social
📅 April 30, 3 PM CEST #OpenScience
Reposted by Isaac Handley-Miner
Some nice new papers hitting the AMPPS website-- check 'em out! @psychscience.bsky.social
April 15, 2025 at 6:08 PM
Some nice new papers hitting the AMPPS website-- check 'em out! @psychscience.bsky.social
Reposted by Isaac Handley-Miner
If you want scholarly publishing to be more aligned with scholarly values, then try out Lifecycle Journal.
If you embrace transparency, then you can do your research in the Lifecycle Journal without risking publishing in the existing system if you later decide to do so. A🧵
lifecyclejournal.org
If you embrace transparency, then you can do your research in the Lifecycle Journal without risking publishing in the existing system if you later decide to do so. A🧵
lifecyclejournal.org
April 15, 2025 at 12:52 PM
If you want scholarly publishing to be more aligned with scholarly values, then try out Lifecycle Journal.
If you embrace transparency, then you can do your research in the Lifecycle Journal without risking publishing in the existing system if you later decide to do so. A🧵
lifecyclejournal.org
If you embrace transparency, then you can do your research in the Lifecycle Journal without risking publishing in the existing system if you later decide to do so. A🧵
lifecyclejournal.org
Reposted by Isaac Handley-Miner
🎥 Missed the 2025 BITSS Annual Meeting? The full event recordings are now on YouTube!
Watch this year's presentations on research transparency and reproducibility below.
www.youtube.com/@berkeleyini...
@tedmiguel.bsky.social @cega-uc.bsky.social @sdellavi.bsky.social
Watch this year's presentations on research transparency and reproducibility below.
www.youtube.com/@berkeleyini...
@tedmiguel.bsky.social @cega-uc.bsky.social @sdellavi.bsky.social
Berkeley Initiative for Transparency in the Social Sciences (BITSS)
The Berkeley Initiative for Transparency in the Social Sciences (BITSS) was established in 2012 to strengthen the quality of social science research and evidence used for policy-making. The initiative...
www.youtube.com
March 19, 2025 at 10:00 PM
🎥 Missed the 2025 BITSS Annual Meeting? The full event recordings are now on YouTube!
Watch this year's presentations on research transparency and reproducibility below.
www.youtube.com/@berkeleyini...
@tedmiguel.bsky.social @cega-uc.bsky.social @sdellavi.bsky.social
Watch this year's presentations on research transparency and reproducibility below.
www.youtube.com/@berkeleyini...
@tedmiguel.bsky.social @cega-uc.bsky.social @sdellavi.bsky.social
Reposted by Isaac Handley-Miner
In the latest blog of the Meta-Research Center, I wonder if meta-scientists should hold themselves to higher standards. Do we lose credibility if we don't always follow all best practices we advocate for?
#metascience #statcheck #psychscisky
metaresearch.nl/blog/2025/3/...
#metascience #statcheck #psychscisky
metaresearch.nl/blog/2025/3/...
Should meta-scientists hold themselves to higher standards? — Meta-Research Center
This blogpost was written by Michèle Nuijten. Michèle is an assistant professor of our research group who currently studies reproducibility and replicability in psychology. She is also the developer o...
metaresearch.nl
March 20, 2025 at 10:00 AM
In the latest blog of the Meta-Research Center, I wonder if meta-scientists should hold themselves to higher standards. Do we lose credibility if we don't always follow all best practices we advocate for?
#metascience #statcheck #psychscisky
metaresearch.nl/blog/2025/3/...
#metascience #statcheck #psychscisky
metaresearch.nl/blog/2025/3/...
Reposted by Isaac Handley-Miner
An excellent investigation highlighting the value of sharing code, in addition to data, for establishing reproducibility.
We crowdsourced 85 teams to indep. computationally reproduce results from a single study in randomized conditions. Reproducibility:
• depends heavily on the transparency of materials
• high (~95%) for same sign/sig, but low (btw. 48-77% depending on transparency) when requiring within 0.1 effects
• depends heavily on the transparency of materials
• high (~95%) for same sign/sig, but low (btw. 48-77% depending on transparency) when requiring within 0.1 effects
March 20, 2025 at 10:20 AM
An excellent investigation highlighting the value of sharing code, in addition to data, for establishing reproducibility.
Reposted by Isaac Handley-Miner
🚀Very excited to finally see our paper on nonsignificance misinterpretations published! 📈
Together, @smurphee.bsky.social, Aurelio Fernández, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos...
Together, @smurphee.bsky.social, Aurelio Fernández, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos...
March 19, 2025 at 1:06 PM
🚀Very excited to finally see our paper on nonsignificance misinterpretations published! 📈
Together, @smurphee.bsky.social, Aurelio Fernández, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos...
Together, @smurphee.bsky.social, Aurelio Fernández, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos...
Reposted by Isaac Handley-Miner
I can't really see any downsides to sharing code -- it's very simple and many concerns that apply to data sharing don't apply at all -- so I really hope we can get that number up all the way, until it's just a no-brainer.
March 17, 2025 at 10:46 AM
I can't really see any downsides to sharing code -- it's very simple and many concerns that apply to data sharing don't apply at all -- so I really hope we can get that number up all the way, until it's just a no-brainer.
Reposted by Isaac Handley-Miner
How can we reduce conceptual clutter in the psychological sciences?
@ruimata.bsky.social and I propose a solution based on a fine-tuned 🤖 LLM (bit.ly/mpnet-pers) and test it for 🎭 personality psychology.
The paper is finally out in @natrevpsych.bsky.social: go.nature.com/4bEaaja
@ruimata.bsky.social and I propose a solution based on a fine-tuned 🤖 LLM (bit.ly/mpnet-pers) and test it for 🎭 personality psychology.
The paper is finally out in @natrevpsych.bsky.social: go.nature.com/4bEaaja
March 11, 2025 at 10:57 AM
How can we reduce conceptual clutter in the psychological sciences?
@ruimata.bsky.social and I propose a solution based on a fine-tuned 🤖 LLM (bit.ly/mpnet-pers) and test it for 🎭 personality psychology.
The paper is finally out in @natrevpsych.bsky.social: go.nature.com/4bEaaja
@ruimata.bsky.social and I propose a solution based on a fine-tuned 🤖 LLM (bit.ly/mpnet-pers) and test it for 🎭 personality psychology.
The paper is finally out in @natrevpsych.bsky.social: go.nature.com/4bEaaja
Reposted by Isaac Handley-Miner
After a long wait, the working paper for the Many-Economists Project: The Sources of Researcher Variation in Economics. We had 146 teams perform the same research three times, each time with less freedom. What source of freedom leads to different choices and results? papers.ssrn.com/sol3/papers....
The Sources of Researcher Variation in Economics
We use a rigorous three-stage many-analysts design to assess how different researcher decisions—specifically data cleaning, research design, and the interpretat
papers.ssrn.com
February 25, 2025 at 7:17 PM
After a long wait, the working paper for the Many-Economists Project: The Sources of Researcher Variation in Economics. We had 146 teams perform the same research three times, each time with less freedom. What source of freedom leads to different choices and results? papers.ssrn.com/sol3/papers....