Jay Patel
@infotainment.bsky.social
defragmenting emotions
#HCI #PeerReview #SciPub
#toolsforthought #ResearchSynthesis
#OpenScience #MetaSci #FoSci
🔎 Research: peer review ethnography
🧑🏫 Teaching: Stats, DataViz
🐢 UMD: College of Info
🌐 PhD Candidate: Info Studies / HCI + Data
🏝️ OASISlab
#HCI #PeerReview #SciPub
#toolsforthought #ResearchSynthesis
#OpenScience #MetaSci #FoSci
🔎 Research: peer review ethnography
🧑🏫 Teaching: Stats, DataViz
🐢 UMD: College of Info
🌐 PhD Candidate: Info Studies / HCI + Data
🏝️ OASISlab
Pinned
Jay Patel
@infotainment.bsky.social
· Jun 11
🌐 It strikes me that crowdsourced evaluation/benchmarking of LLMs for scientific work would be valuable and possible in the style of ManyLabs projects.
If I coordinate crowds of volunteers to assess how well LLMs can evaluate papers, would you be interested?
#openscience #metascience #AI #LLM
If I coordinate crowds of volunteers to assess how well LLMs can evaluate papers, would you be interested?
#openscience #metascience #AI #LLM
A nice reform to support the publication or replications (a major issue):
1. plan and submit study to journal as replication
2. review
3. accept/reject based on the idea, not data
4. collect data
5. complete paper and submit for acceptance
Does this make doing replications more palatable to you?
1. plan and submit study to journal as replication
2. review
3. accept/reject based on the idea, not data
4. collect data
5. complete paper and submit for acceptance
Does this make doing replications more palatable to you?
New submission format at SBE:
“Replications as Registered Reports”
link.springer.com/journal/1118...
You can get "in-principle acceptance" before data collection even begins; final paper gets published regardless the results, if the study is conducted rigorously.
#EconSky
“Replications as Registered Reports”
link.springer.com/journal/1118...
You can get "in-principle acceptance" before data collection even begins; final paper gets published regardless the results, if the study is conducted rigorously.
#EconSky
January 29, 2026 at 4:15 PM
A nice reform to support the publication or replications (a major issue):
1. plan and submit study to journal as replication
2. review
3. accept/reject based on the idea, not data
4. collect data
5. complete paper and submit for acceptance
Does this make doing replications more palatable to you?
1. plan and submit study to journal as replication
2. review
3. accept/reject based on the idea, not data
4. collect data
5. complete paper and submit for acceptance
Does this make doing replications more palatable to you?
Reposted by Jay Patel
👨🏼🎓 Me: Stews over the Theoretical Contributions section of the Discussion for months in quiet contemplation. Writing, rewriting, doubting, never fully sure.
🤖 ML researchers: "We offer no explanation as to why these architectures seem to work; we attribute their success... to divine benevolence."
🤖 ML researchers: "We offer no explanation as to why these architectures seem to work; we attribute their success... to divine benevolence."
January 28, 2026 at 2:40 AM
👨🏼🎓 Me: Stews over the Theoretical Contributions section of the Discussion for months in quiet contemplation. Writing, rewriting, doubting, never fully sure.
🤖 ML researchers: "We offer no explanation as to why these architectures seem to work; we attribute their success... to divine benevolence."
🤖 ML researchers: "We offer no explanation as to why these architectures seem to work; we attribute their success... to divine benevolence."
I thought my #AcademicWriting was bold and spicy 🌶️
In this 🤖AI preprint, I find an example that asks me to level up my game:
“Karl Marx wrote that “the human essence is the ensemble of social relations”
In this 🤖AI preprint, I find an example that asks me to level up my game:
“Karl Marx wrote that “the human essence is the ensemble of social relations”
January 26, 2026 at 3:55 PM
I thought my #AcademicWriting was bold and spicy 🌶️
In this 🤖AI preprint, I find an example that asks me to level up my game:
“Karl Marx wrote that “the human essence is the ensemble of social relations”
In this 🤖AI preprint, I find an example that asks me to level up my game:
“Karl Marx wrote that “the human essence is the ensemble of social relations”
Reposted by Jay Patel
Whereas retractions usually expose scandals in science, citations to retracted articles are a bigger scandal, especially when they are post-retraction.
XeraRetractions web app: openscience.xera.ac/retractions
XeraRetractions web app: openscience.xera.ac/retractions
XeraRetractionTracker - Retracted Paper Database & Citation Tracker
Discover which papers cite retracted research. Comprehensive database updated daily with Retraction Watch data. Essential tool for research integrity.
openscience.xera.ac
January 25, 2026 at 1:45 AM
Whereas retractions usually expose scandals in science, citations to retracted articles are a bigger scandal, especially when they are post-retraction.
XeraRetractions web app: openscience.xera.ac/retractions
XeraRetractions web app: openscience.xera.ac/retractions
Reposted by Jay Patel
Now that almost every social media system in the US is owned by like, 3 people, the existence of open algorithms and protocols becomes totally critical
January 23, 2026 at 5:12 PM
Now that almost every social media system in the US is owned by like, 3 people, the existence of open algorithms and protocols becomes totally critical
This one's a first for me.
The paper did, in fact, contain data.
The paper did, in fact, contain data.
January 22, 2026 at 3:40 AM
This one's a first for me.
The paper did, in fact, contain data.
The paper did, in fact, contain data.
Reason #92893 for why **scholars** need to own scholarly infrastructure.
Fed up, author issues her own retraction after journal ghosts her
"She documented the self-retraction on Figshare... she “unilaterally retracts” the article for “persistent non-responsiveness of the publisher”
Fed up, author issues her own retraction after journal ghosts her
"She documented the self-retraction on Figshare... she “unilaterally retracts” the article for “persistent non-responsiveness of the publisher”
Fed up, author issues her own retraction after journal ghosts her
At wit’s end after a publisher ignored her repeated requests for a correction, Ursula Bellut-Staeck took the extreme step of issuing her own retraction. But is that even a thing? Bellut…
retractionwatch.com
January 21, 2026 at 11:03 PM
Reason #92893 for why **scholars** need to own scholarly infrastructure.
Fed up, author issues her own retraction after journal ghosts her
"She documented the self-retraction on Figshare... she “unilaterally retracts” the article for “persistent non-responsiveness of the publisher”
Fed up, author issues her own retraction after journal ghosts her
"She documented the self-retraction on Figshare... she “unilaterally retracts” the article for “persistent non-responsiveness of the publisher”
Reposted by Jay Patel
We already have several speakers and volunteers in need of travel support to attend #ATScience - if you’re able to pitch in with event sponsorship, please be in touch with us here or hello@atproto.science! Tons of great energy swirling around ATScience, let’s make it the best possible for everyone!
January 13, 2026 at 2:56 PM
We already have several speakers and volunteers in need of travel support to attend #ATScience - if you’re able to pitch in with event sponsorship, please be in touch with us here or hello@atproto.science! Tons of great energy swirling around ATScience, let’s make it the best possible for everyone!
Reposted by Jay Patel
New podcast klaxon!
I'm launching a podcast called Under Review that explores academia's wicked problems.
My esteemed guests are going to get the chance to turn academia off and back on again as well as decide what goes into the Under Review waste paper bin.
www.youtube.com/watch?v=anuj...
I'm launching a podcast called Under Review that explores academia's wicked problems.
My esteemed guests are going to get the chance to turn academia off and back on again as well as decide what goes into the Under Review waste paper bin.
www.youtube.com/watch?v=anuj...
Launching the Under Review Podcast
YouTube video by Andy Tattersall
www.youtube.com
January 20, 2026 at 3:57 PM
New podcast klaxon!
I'm launching a podcast called Under Review that explores academia's wicked problems.
My esteemed guests are going to get the chance to turn academia off and back on again as well as decide what goes into the Under Review waste paper bin.
www.youtube.com/watch?v=anuj...
I'm launching a podcast called Under Review that explores academia's wicked problems.
My esteemed guests are going to get the chance to turn academia off and back on again as well as decide what goes into the Under Review waste paper bin.
www.youtube.com/watch?v=anuj...
Reposted by Jay Patel
Looking for user panelists to contribute to the Stage 2 of our registered report: Developing and Piloting a QUick Evaluation of Research Integrity for Evidence Synthesis in Educational Research (QUERIES-Edu). osf.io/tc5n3
If you’re interested, sign up here: forms.gle/SSmPmaCrdGvnHPdx9
If you’re interested, sign up here: forms.gle/SSmPmaCrdGvnHPdx9
January 20, 2026 at 9:40 AM
Looking for user panelists to contribute to the Stage 2 of our registered report: Developing and Piloting a QUick Evaluation of Research Integrity for Evidence Synthesis in Educational Research (QUERIES-Edu). osf.io/tc5n3
If you’re interested, sign up here: forms.gle/SSmPmaCrdGvnHPdx9
If you’re interested, sign up here: forms.gle/SSmPmaCrdGvnHPdx9
Reposted by Jay Patel
Adversarial collabs. aside,
1. Do you/colleagues participate in *agnostic* or *consulting* collabs.?
2. Do they work well?
Example: Scholar A helps Scholar B test something using methods more familiar to A even tho A doesn't have an ax to grind WRT B's literature/debate.
1. Do you/colleagues participate in *agnostic* or *consulting* collabs.?
2. Do they work well?
Example: Scholar A helps Scholar B test something using methods more familiar to A even tho A doesn't have an ax to grind WRT B's literature/debate.
November 7, 2024 at 11:57 PM
Adversarial collabs. aside,
1. Do you/colleagues participate in *agnostic* or *consulting* collabs.?
2. Do they work well?
Example: Scholar A helps Scholar B test something using methods more familiar to A even tho A doesn't have an ax to grind WRT B's literature/debate.
1. Do you/colleagues participate in *agnostic* or *consulting* collabs.?
2. Do they work well?
Example: Scholar A helps Scholar B test something using methods more familiar to A even tho A doesn't have an ax to grind WRT B's literature/debate.
Reposted by Jay Patel
Serious question: I've been reviewing journal submissions for <20 years but this is the first time that I am reading something so clearly duplicitous that I feel that the authors' institutions should be contacted. Is there ever a reason or way to do this?
January 21, 2026 at 4:27 PM
Serious question: I've been reviewing journal submissions for <20 years but this is the first time that I am reading something so clearly duplicitous that I feel that the authors' institutions should be contacted. Is there ever a reason or way to do this?
Should the requirements for a self-citation be higher than for a citation to another scholar?
January 20, 2026 at 9:51 PM
Should the requirements for a self-citation be higher than for a citation to another scholar?
Before the jingle-jangle fallacy, there were cyberneticians:
"There are fields of scientific work... in which every single notion receives a separate name from each group, and in which important work has been triplicated..."
Weiner, Cybernetics (1948)
"There are fields of scientific work... in which every single notion receives a separate name from each group, and in which important work has been triplicated..."
Weiner, Cybernetics (1948)
January 20, 2026 at 2:09 AM
Before the jingle-jangle fallacy, there were cyberneticians:
"There are fields of scientific work... in which every single notion receives a separate name from each group, and in which important work has been triplicated..."
Weiner, Cybernetics (1948)
"There are fields of scientific work... in which every single notion receives a separate name from each group, and in which important work has been triplicated..."
Weiner, Cybernetics (1948)
What's stopping you from leaving comments on preprints and papers somewhere online?
- fear?
- time?
- energy?
- fear?
- time?
- energy?
One thing I want to get better at is giving feedback on preprints (both constructive and plainly positive). There are lots of times where I have something to formulated in my head to share to authors but never actually get around to doing it. I just gotta write it down and press send.
January 20, 2026 at 1:25 AM
What's stopping you from leaving comments on preprints and papers somewhere online?
- fear?
- time?
- energy?
- fear?
- time?
- energy?
Reposted by Jay Patel
One thing I want to get better at is giving feedback on preprints (both constructive and plainly positive). There are lots of times where I have something to formulated in my head to share to authors but never actually get around to doing it. I just gotta write it down and press send.
July 30, 2025 at 1:16 PM
One thing I want to get better at is giving feedback on preprints (both constructive and plainly positive). There are lots of times where I have something to formulated in my head to share to authors but never actually get around to doing it. I just gotta write it down and press send.
Reposted by Jay Patel
Sign up now for the Perspectives on Scientific Error Conference (PSE) on 11 - 13 Feb in Leiden, the Netherlands
We will be hosting a mentor-mentee lunch during the conference. Sign up, and you will be matched to another academic based on shared research interests!
Don't miss out! See you there 🙌
We will be hosting a mentor-mentee lunch during the conference. Sign up, and you will be matched to another academic based on shared research interests!
Don't miss out! See you there 🙌
PSE8
Visit the post for more.
tinyurl.com
January 12, 2026 at 8:32 AM
Sign up now for the Perspectives on Scientific Error Conference (PSE) on 11 - 13 Feb in Leiden, the Netherlands
We will be hosting a mentor-mentee lunch during the conference. Sign up, and you will be matched to another academic based on shared research interests!
Don't miss out! See you there 🙌
We will be hosting a mentor-mentee lunch during the conference. Sign up, and you will be matched to another academic based on shared research interests!
Don't miss out! See you there 🙌
If you need more evidence that developing acronyms in papers is inadvisable for most, here it is:
"interpersonal brain synchronization (IBS)." 😬
Maybe stick to the full name instead.
"interpersonal brain synchronization (IBS)." 😬
Maybe stick to the full name instead.
January 19, 2026 at 8:15 PM
If you need more evidence that developing acronyms in papers is inadvisable for most, here it is:
"interpersonal brain synchronization (IBS)." 😬
Maybe stick to the full name instead.
"interpersonal brain synchronization (IBS)." 😬
Maybe stick to the full name instead.
Crowdsourcing IMDB content sure has its disadvantages. Gotta have some kind of filter for the snarky commenters posting Trivia like:
"Eline is trying to destroy the art of acting by her persistent insistence of using Al Generated actors instead of real-life actors."
"Eline is trying to destroy the art of acting by her persistent insistence of using Al Generated actors instead of real-life actors."
January 18, 2026 at 9:12 PM
Crowdsourcing IMDB content sure has its disadvantages. Gotta have some kind of filter for the snarky commenters posting Trivia like:
"Eline is trying to destroy the art of acting by her persistent insistence of using Al Generated actors instead of real-life actors."
"Eline is trying to destroy the art of acting by her persistent insistence of using Al Generated actors instead of real-life actors."
Shoutout to the heart-warming shoutout in this Acknowledgements section:
"We also thank Carolyn Gilbert, who in no way helped this paper get written, but is wonderful so she gets mentioned anyway."
"We also thank Carolyn Gilbert, who in no way helped this paper get written, but is wonderful so she gets mentioned anyway."
January 17, 2026 at 2:51 AM
Shoutout to the heart-warming shoutout in this Acknowledgements section:
"We also thank Carolyn Gilbert, who in no way helped this paper get written, but is wonderful so she gets mentioned anyway."
"We also thank Carolyn Gilbert, who in no way helped this paper get written, but is wonderful so she gets mentioned anyway."
The world today, in a nutshell.
h/t the other site
h/t the other site
January 16, 2026 at 10:56 PM
The world today, in a nutshell.
h/t the other site
h/t the other site
Reposted by Jay Patel
"Credit in research goes hand in hand with responsibility" www.nature.com/articles/d41... Hmm. How often have you seen a big Nature or Science paper with 20 authors, who all get to enjoy the credit, but if it gets retracted it turns out the first author was solely responsible?
Credit in research goes hand in hand with responsibility
Trust in science needs researchers, journals and institutions to correct the scientific record quickly and transparently when errors are found.
www.nature.com
January 16, 2026 at 4:47 PM
"Credit in research goes hand in hand with responsibility" www.nature.com/articles/d41... Hmm. How often have you seen a big Nature or Science paper with 20 authors, who all get to enjoy the credit, but if it gets retracted it turns out the first author was solely responsible?
Reposted by Jay Patel
If people expert in research methods publicise lists of research they have assessed as adequate and at least potentially significant, that might help focus attention on deserving work, and less on work that's very-difficult-to-assess-regardless-of-expertise.
bsky.app/profile/matt...
bsky.app/profile/matt...
In a field where evidence suggests that a high proportion of papers have serious flaws, a database of "potentially important" papers which appear *not* to have serious flaws might be useful, as well as identifying those that do.
January 15, 2026 at 6:39 PM
If people expert in research methods publicise lists of research they have assessed as adequate and at least potentially significant, that might help focus attention on deserving work, and less on work that's very-difficult-to-assess-regardless-of-expertise.
bsky.app/profile/matt...
bsky.app/profile/matt...
Reposted by Jay Patel
"The rewards to 'discovering' a spectacular scientific finding [in psychology] are large; the rewards to debunking frauds or deflating exaggerated claims are small if not non-existent. If these are the rules of the game, we should not be surprised at the way the game is played."
When psychologists mislead us
From Piltdown Man to the Stanford prison experiment, many famous scientific discoveries have been exposed as hoaxes or distortions
www.ft.com
January 15, 2026 at 3:51 PM
"The rewards to 'discovering' a spectacular scientific finding [in psychology] are large; the rewards to debunking frauds or deflating exaggerated claims are small if not non-existent. If these are the rules of the game, we should not be surprised at the way the game is played."