#MetaSci
LightNews LightNews
devezer.bsky.social
#metasci #statsky
November 10, 2025 at 5:33 PM
markrubin.bsky.social
"Basic scientists consistently enhance scientific output quality, yet research and newly joining researchers have shifted toward applied orientations."

Preprint: arxiv.org/abs/2509.01530

#PhilSci #MetaSci #AcademicSky 🧪
Despite broad acclaim for basic research, science is undergoing an applied shift that marginalizes basic scientists. This gap reflects an incomplete understanding of their distinctive roles, which prevents translating philosophical appreciation into effective support. We introduce a scalable metric--the application score--to position research along the basic-applied spectrum and apply it to 62 million publications (1970-2023) to reveal the distinctive contributions of basic scientists. We find a structural asymmetry: involvement of basic scientists substantially increases citation impact, even more so in applied contexts, while applied scientists show no such effect in basic domains. This asymmetric effect arises from their intellectual leadership in conceptualization, writing, and experimental design, amplified in large, multidisciplinary, and intermediate career teams. Yet basic scientists remain concentrated in historically prestigious institutions, while new entrants shift toward applied work, indicating critical undersupply. These findings provide large-scale evidence for the indispensable role of basic scientists, guiding policy and institutional strategy to sustain the foundations of discovery and innovation.
November 8, 2025 at 11:37 AM
khinsen.net
Revised preprint: "Establishing trust in automated reasoning"

osf.io/preprints/me...

Based on two very helpful reviews from @metaror.bsky.social:

metaror.org/kotahi/artic...

More references, better discussion of unfamiliar concepts (e.g. conviviality), and more.

🧪 #metasci #compsci
OSF
osf.io
November 6, 2025 at 9:26 AM
devezer.bsky.social
Here's a link to the paper. Let me know what you think.

#metasci #philsci #statsky #sts

11/11
pmc.ncbi.nlm.nih.gov
November 5, 2025 at 7:07 PM
markrubin.bsky.social
“[Psychologists] were typically interested in going beyond direct replication and exploring the phenomenon further in conceptual replications. Thus, they seemed to entertain the kind of wider view of replication that Feest and others have emphasized.”

#MetaSci #STS #OpenSci #AcademicSky
markrubin.bsky.social Mark Rubin @markrubin.bsky.social · May 25
“Why replication is overrated” by @ufeest.bsky.social

doi.org/10.1086/705451
Current debates about the replication crisis in psychology take it for granted that direct replication is valuable, largely focusing on its role in uncovering questionable statistical practices. This article takes a broader look at the notion of replication in psychological experiments. It is argued that all experimentation/replication involves individuation judgments and that research in experimental psychology frequently turns on probing the adequacy of such judgments. In this vein, I highlight the ubiquity of conceptual and material questions in research, arguing that replication has its place but is not as central to psychological research as it is sometimes taken to be.
November 1, 2025 at 2:01 PM
wamsleylab.bsky.social
So, apparently, "left-wing bias" is responsible for a failure of sociologists to adopt #openscience practices and robust causal inference methods. 🤔 And apparently, this implies that enforcing political "viewpoint diversity" among the faculty is a "scientific necessity."

🫤

#academicsky #metasci
Opinion | Left-Wing Bias Is Corrupting Sociology
Other social scientists have reformed. Sociologists haven’t.
www.chronicle.com
October 23, 2025 at 1:19 AM
markrubin.bsky.social
“Multiverse analyses require thorough, theory-based model selection. Otherwise, they become a ‘dangerous tool’ that drowns valid models in misspecified ones, needlessly eroding trust in science.”

A plea for thoughtful models by @kauspurg.bsky.social

#MetaSci
October 21, 2025 at 4:18 PM
markrubin.bsky.social
"Disadvantaged members of society might perceive experts as members of the elite that they feel alienated from and are thus less inclined to leave deliberative power in their hands."

#MetaSci #AcademicSky #SciComm #SocialPsyc 🧪
September 25, 2025 at 5:20 AM
markrubin.bsky.social
Multiverse Analyses

Nice consideration of five nonmutually exclusive motives for conducting multiverse analyses:

🔹️ Assessing robustness

🔹️ Examining boundary conditions

🔹️ Generating hypotheses

🔹️ Increasing transparency

🔹️ Improving a study’s methodology

#MetaSci #Methodology
When processing and analyzing empirical data, researchers regularly face choices that may appear arbitrary (e.g., how to define and handle outliers). If one chooses to exclusively focus on a particular option and conduct a single analysis, its outcome might be of limited utility. That is, one remains agnostic regarding the generalizability of the results, because plausible alternative paths remain unexplored. A multiverse analysis offers a solution to this issue by exploring the various choices pertaining to data processing and/or model building, and examining their impact on the conclusion of a study. However, even though multiverse analyses are arguably less susceptible to biases compared to the typical single-pathway approach, it is still possible to selectively add or omit pathways. To address this issue, we outline a novel, more principled approach to conducting multiverse analyses through crowdsourcing. The approach is detailed in a step-by-step tutorial to facilitate its implementation. We also provide a worked out illustration featuring the Semantic Priming Across Many Languages project, thereby demonstrating its feasibility and its ability to increase objectivity and transparency.
September 23, 2025 at 7:14 AM
reproducibilitea.org
Join our upcoming webinars “Peer Review and its Diversification”!

📅 15-16 October 2025 on Zoom

👉 Registration: forms.gle/1R4Y3d33W8bQ...

📄 Full programme: docs.google.com/document/d/1...

#PeerReview #PeerReviewWeek #PRW2025 #metasci #academicsky
September 18, 2025 at 3:41 PM
markrubin.bsky.social
📣 Call for Papers 📣

New collection in Synthese on:

"Research Integrity, Norms of Good Practices, and Misconduct across Scientific Fields: A Philosophical Perspective"

link.springer.com/collections/...

BSky Guest Editor: @aurelienallard.bsky.social

#PhilSci #MetaSci #Methodology #Epistemology
March 19, 2024 at 7:38 AM
markrubin.bsky.social
“Heuristics for metascience: Simon and Popper”

Katsikopoulos et al.’s chapter in Gigerenzer et al.’s (2024) “Elgar Companion to Herbert Simon”

doi.org/10.4337/9781...

#MetaSci #PhilSci
July 16, 2024 at 6:25 AM
annaalexandrova.bsky.social
So glad to see this work in a science journal! @erantal.bsky.social congratulations to Sebastian, you, and Skye. May this article be read far and wide. #philsci #philsky #scipol #metascience #openScience #psychscisky #psychology #methodology #MetaSci
December 9, 2023 at 3:24 PM
markrubin.bsky.social
New preprint discusses three theory-based QRPs:

🔹 Hypothesizing before understanding theory

🔹 Liberal theory interpretation

🔹 Hypothesis cherry picking

ecoevorxiv.org/repository/v...

#MetaSci #PhilSci #Methodology #evosky 🧪
Questionable research practices, QRPs, arising from a theory crisis (via epistemological issues) and replication crisis (via methodological issues) and how they relate to research waste. QRPs concerning the theory crisis relate to poor conceptual design, which comes upstream of the other 3 items of research waste related to QRPs resulting from a replication crisis. Note that selective reporting and incomplete reporting may sound similar, but the former indicates deliberate selection of positive results while the latter represents the lack of culture in providing all the results and associated outcomes, including associated data and code.
June 4, 2024 at 12:00 PM
markrubin.bsky.social
@deevybee.bsky.social’s excellent presentation at Durham University's RIOTS Club today on scientific fraud and how to detect it, including a discussion of the ingredients of a Gobbledegook Sandwich article!

#MetaSci #Methodology
May 28, 2024 at 11:22 AM
wamsleylab.bsky.social
"When to cite an NIH Grant in a Paper?"

Comments on this NIH post about overciting grant support for unrelated work are worth a read.

A lot of grey territory, but for sure there are some perverse incentives leading to inaccurate reporting.

nexus.od.nih.gov/all/2023/10/...

#Academicsky #metasci
Reminders About When to Cite an NIH Grant in a Paper: Overcite Oversight
In 2021, we wrote that appropriately acknowledging NIH grant support allows us to properly assess award outputs and make recommendations for future research directions. This post revisits the issue as...
nexus.od.nih.gov
October 20, 2023 at 12:37 PM
iai.tv
“Though the changes in complexity may be small, we have seen that similarly small changes in other neuroimaging measures have a massive effect on subjective experience,” argues David Nutt. | https://buff.ly/40kNr6u

Tap to read more. 🧪 #philsci #metasci
David Nutt: entropy explains consciousness
buff.ly
January 17, 2025 at 8:00 AM
markrubin.bsky.social
Open Science and Ethnography

“Claims that open data would improve data quality are, in our assessment, potentially reversed: in our own ethnographic work, open data practices would likely have compromised our data quality.”

Few quotes 🧵

#MetaSci #PhilSci #STS #Methodology #Ethnography #Sociology
julianhamann.bsky.social Julian Hamann @julianhamann.bsky.social · Mar 2
Important paper by Shamus Khan et al. on relation between open data and qualitative work. Epistemologies of qualitative research don't always align with open data. Open data movement subsumes different epistemic cultures under a singular scientific approach.
www.frontiersin.org/articles/10....
Screenshot of the article's title page, showing title, authors, and abstract
March 2, 2024 at 4:59 PM
asmiley.bsky.social
#AcademicSky #MetaSci #StatsSky #PsychSciSky
November 22, 2023 at 1:37 PM
markrubin.bsky.social
Replication Failures can be Good!

Really interesting new paper by John Sweller argues that replication failures can have positive effects vis-à-vis theory development.

Open Access: doi.org/10.1007/s106...

#PhilSci #MetaSci #EduSky #PsychSciSky

Couple of quotes follow: 🧵👉
September 21, 2023 at 10:08 AM
fraor.bsky.social
After talking to ChatGPT I sometimes feel like that blackboard Bart Simpson writes on.

#Chatgpt #AI #Psychology #MachineLearning #NeurosymbolicAI
#AIarchitecture #MetaSci #Methodology #PhilSci #GPT5 #GPT4o #fraor #LLM #Alsky #rstats
August 30, 2025 at 10:28 AM
wamsleylab.bsky.social
Excellent visual explainer article on the flaws of David Geier's anti-vax pseudoscience.

Will have my psych research methods class read this -- hammers home the real-world meaning of poorly designed controls, lax ethics, bias, replication.

#metasci

Gift link: www.nytimes.com/interactive/...
Image from the article, in cartoon style, showing Geiers plucking children out of a line to receive vaccines.
September 6, 2025 at 12:16 PM
fraor.bsky.social
The paradox of any system is that it is created to contain chaos, but over time it itself becomes its source when it forgets why it was created.

#philsci #histsci #philsky #MetaSci #Methodology #Psychology #Philosophy
October 27, 2025 at 3:00 AM
gordonhodsonphd.bsky.social
I realize some are chuckling about this "size matters" paper (re journalists evaluate science largely on sample size).

journals.sagepub.com/doi/10.1177/...

But this is worrying. Clearly we're not doing a good enough job explaining how to evaluate science to consumers.

#PsychSciSky #metasci
October 23, 2023 at 6:54 PM
devezer.bsky.social
Intriguing post. I don't agree with everything Munger says, nor want to discuss specific points that seem to be of personal nature. But some of his observations about #metasci are on point, such as the distinction btw wanting to persuade colleagues vs imposing how everyone needs to do science. >
Do Something Else
Lakens and the Second Reformation
kevinmunger.substack.com
September 23, 2024 at 10:20 PM