Sean Harrison, PhD
@sean-h.bsky.social
Pinned
Sean Harrison, PhD
@sean-h.bsky.social
· Jun 14
I was saving this for an article, but I want something to refer to when I talk about how I think the best way to fix peer-review is to abolish journals.
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
December 1, 2025 at 9:23 AM
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
This is what happens when as a single condition health charity you’re overly focused on those who die of your condition and you don’t consider outcomes for those who don’t, or the opportunity costs involved for NHS. PCUK are particularly guilty but they’re not alone.
www.thetimes.com/article/4487...
www.thetimes.com/article/4487...
Start prostate cancer testing regardless of advice, Wes Streeting told
NHS advisers are expected to rule against a mass programme for the disease, which causes more than 12,000 deaths a year
www.thetimes.com
November 28, 2025 at 1:55 PM
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
one of my most contrarian thoughts is to tax IHT at 100% - & use the money to lower tax on actually working for a living
November 24, 2025 at 3:18 PM
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Most scientists don't understand how effect sizes work and are therefore far too quick to dismiss "small" effects.
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
November 24, 2025 at 9:12 AM
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Reposted by Sean Harrison, PhD
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
October 9, 2025 at 4:01 PM
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Y'all. I just got ChatGPT to do everything in R for this manuscript. I mean EVERYTHING. And it's all legit and reproducible. I'm shook.
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
Statistics in the era of AI
How do we mentor, teach, and do stats when AI can do so much of the work?
scienceforeveryone.science
October 9, 2025 at 3:10 PM
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
I read close to 50 papers in order to give an evidence-based, accurate response to Monday's press conference.
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
All About the Tylenol-Autism Brouhaha
At an unhinged press conference last Monday, Donald Trump and his medical acolytes revealed an actionable cause of autism at long last: Tylenol. In what felt like a social media post come to life, Tru...
www.mcgill.ca
September 29, 2025 at 9:57 AM
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Can large language models stand in for human participants?
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
The threat of analytic flexibility in using large language models to simulate human data: A call to attention
Social scientists are now using large language models to create "silicon samples" - synthetic datasets intended to stand in for human respondents, aimed at revolutionising human subjects research. How...
arxiv.org
September 18, 2025 at 9:29 AM
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Reposted by Sean Harrison, PhD
John Finnemore's Souvenir Programme back for 2025
John Finnemore's Souvenir Programme back for 2025
The best of the week's comedy on TV, radio and streaming
dlvr.it
August 24, 2025 at 10:03 AM
John Finnemore's Souvenir Programme back for 2025
p=1, bitches
New rule: Not engaging with people who can not understand that when you randomize, the pre-experiment population means in each group are the same and that any differences are due to sampling variability.
August 16, 2025 at 6:46 PM
p=1, bitches
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
August 15, 2025 at 12:38 PM
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
this is excellent. some junior colleagues keep thinking high IF = good paper, and im trying to explain repeatedly that this is not a good way to appraise a publication
This is an excellent illustration of how useless the "Impact Factor" metric is.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
August 14, 2025 at 11:03 AM
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
Reposted by Sean Harrison, PhD
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...
August 12, 2025 at 9:54 AM
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...
To win a bet, I once flipped a random coin I had in my wallet 20 times, and had it land on heads every time.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
I don't understand much of the stats here or probably all of the implications, but I love this! Great work @fbartos.bsky.social and all.
I also hope there are no triallists still clinging on to coin-flipping as a means of randomisation...
I also hope there are no triallists still clinging on to coin-flipping as a means of randomisation...
Fair coins tend to land on the same side they started: evidence from 350,757 flips.
That's the title of our paper summarizing ~650 hours of coin-tossing experimentation just published in the Journal of the American Statistical Association.
doi.org/10.1080/0162...
That's the title of our paper summarizing ~650 hours of coin-tossing experimentation just published in the Journal of the American Statistical Association.
doi.org/10.1080/0162...
August 11, 2025 at 8:40 PM
To win a bet, I once flipped a random coin I had in my wallet 20 times, and had it land on heads every time.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
Think I won 50p.
With the right flipping technique, one could absolutely dictate the outcome.
With hindsight, I probably should be used that skill to make a heap of money...
Oh well.
The person who wrote this list seems to have no idea either what each of these jobs entails or the limits of LLMs.
tbf it could be an AI-generated list.
That'd be on-brand.
tbf it could be an AI-generated list.
That'd be on-brand.
Generative AI is the biggest bubble for some time. It can help with certain tasks, but my experience is that it cannot do even the most basic things unsupervised and regularly fabricates its work.
August 5, 2025 at 8:59 AM
The person who wrote this list seems to have no idea either what each of these jobs entails or the limits of LLMs.
tbf it could be an AI-generated list.
That'd be on-brand.
tbf it could be an AI-generated list.
That'd be on-brand.
Reposted by Sean Harrison, PhD
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
The Uncertain Origins of Aspirin
The history of humanity’s pharmacopeia is often muddied by folklore. What can the origins of aspirin teach us about separating fact from fiction?
press.asimov.com
July 14, 2025 at 4:23 PM
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
There was Guardian opinion piece about remove age limits from voting entirely: www.theguardian.com/books/2025/j...
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
July 17, 2025 at 11:15 AM
There was Guardian opinion piece about remove age limits from voting entirely: www.theguardian.com/books/2025/j...
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
I thought about it, and can't come up with objections to removing the voting age that don't equally apply to older groups.
Ha, possibly a true dichotomy:
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
Britons are more regular swearers than Australians or Americans, but what are their top swear words?
Britons
F*ck: 67% use regularly
Sh*t: 65%
Bloody: 54%
Australians
Sh*t: 61%
F*ck: 56%
Bullsh*t: 52%
Americans
Damn: 55%
Sh*t: 54%
F*ck: 47%
👇 more in chart below
yougov.co.uk/society/arti...
Britons
F*ck: 67% use regularly
Sh*t: 65%
Bloody: 54%
Australians
Sh*t: 61%
F*ck: 56%
Bullsh*t: 52%
Americans
Damn: 55%
Sh*t: 54%
F*ck: 47%
👇 more in chart below
yougov.co.uk/society/arti...
July 15, 2025 at 9:58 AM
Ha, possibly a true dichotomy:
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
67% of Britons use "Fuck" regularly
33% of Britons "Have a negative view of swearing"
Possible for overlap, of course, but the numbers line up so well!
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
The Uncertain Origins of Aspirin
The history of humanity’s pharmacopeia is often muddied by folklore. What can the origins of aspirin teach us about separating fact from fiction?
press.asimov.com
July 14, 2025 at 4:23 PM
I wrote a thing: press.asimov.com/articles/asp...
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
If you're interested in whether willow bark = aspirin, it might be interesting!
If not, it still might be interesting!
I have a history with wasps.
When I was a very young child, one was in my wellie, and stung me repeatedly (presumably after being mildly crushed).
When I was slightly older, one stung me repeatedly in bed. No one believed me until they found the wasp in a crumpled heap under the covers.
When I was a very young child, one was in my wellie, and stung me repeatedly (presumably after being mildly crushed).
When I was slightly older, one stung me repeatedly in bed. No one believed me until they found the wasp in a crumpled heap under the covers.
July 7, 2025 at 1:21 PM
I have a history with wasps.
When I was a very young child, one was in my wellie, and stung me repeatedly (presumably after being mildly crushed).
When I was slightly older, one stung me repeatedly in bed. No one believed me until they found the wasp in a crumpled heap under the covers.
When I was a very young child, one was in my wellie, and stung me repeatedly (presumably after being mildly crushed).
When I was slightly older, one stung me repeatedly in bed. No one believed me until they found the wasp in a crumpled heap under the covers.
Unless there's an immensely good reason, don't combine studies with a 26x difference in SD.
They're almost certainly not measuring the same thing, so the meta-analysis results are meaningless.
(on top of interrogate insane results)
They're almost certainly not measuring the same thing, so the meta-analysis results are meaningless.
(on top of interrogate insane results)
Sometimes this stuff just gives me a headache.
A meta-analysis finds a Cohen's d of nearly 11. Nowhere in the paper do the authors interrogate this study, it's barely mentioned.
Just...no.
A meta-analysis finds a Cohen's d of nearly 11. Nowhere in the paper do the authors interrogate this study, it's barely mentioned.
Just...no.
June 24, 2025 at 6:52 AM
Unless there's an immensely good reason, don't combine studies with a 26x difference in SD.
They're almost certainly not measuring the same thing, so the meta-analysis results are meaningless.
(on top of interrogate insane results)
They're almost certainly not measuring the same thing, so the meta-analysis results are meaningless.
(on top of interrogate insane results)
Ignoring the criticism about how this can possibly help in doing things with Government policy, I did enjoy solving the puzzle.
Answer below for anyone interested - I'll put it in the alt text so not immediately obvious.
It'd help to draw it, but that would give spoilers...
Answer below for anyone interested - I'll put it in the alt text so not immediately obvious.
It'd help to draw it, but that would give spoilers...
June 21, 2025 at 12:50 PM
Ignoring the criticism about how this can possibly help in doing things with Government policy, I did enjoy solving the puzzle.
Answer below for anyone interested - I'll put it in the alt text so not immediately obvious.
It'd help to draw it, but that would give spoilers...
Answer below for anyone interested - I'll put it in the alt text so not immediately obvious.
It'd help to draw it, but that would give spoilers...
I think my problem is that LLMs don't know anything, it's just word probability.
Same as politicians and other people who are absolutely happy to say anything with no thought about the meaning of their words.
In both cases, people believe them.
But there's no meaning behind any of it. Only words.
Same as politicians and other people who are absolutely happy to say anything with no thought about the meaning of their words.
In both cases, people believe them.
But there's no meaning behind any of it. Only words.
Maybe the problem is that I just really really hate being lied to.
June 19, 2025 at 6:56 PM
I think my problem is that LLMs don't know anything, it's just word probability.
Same as politicians and other people who are absolutely happy to say anything with no thought about the meaning of their words.
In both cases, people believe them.
But there's no meaning behind any of it. Only words.
Same as politicians and other people who are absolutely happy to say anything with no thought about the meaning of their words.
In both cases, people believe them.
But there's no meaning behind any of it. Only words.