Joe Bak-Coleman
@jbakcoleman.bsky.social
Research Scientist at the University of Washington based in Brooklyn. Also: SFI External Applied Fellow, Harvard BKC affiliate. Collective Behavior, Statistics, etc..
The author's should ask ChatGPT to provide feedback on the risks of nonresponse bias in their design.
If you're trying to design-bias a study towards LLM-friendly findings, this is how you do it.
If you're trying to design-bias a study towards LLM-friendly findings, this is how you do it.
somebody at UChicago is feeding preprints to LLMs without authors' consent, in a research study
they have the gall to suggest to authors they've opted-in that they volunteer to evaluate the LLMs' suggestions regarding their own work.
lol, lmao even. here is the invite and my reply
they have the gall to suggest to authors they've opted-in that they volunteer to evaluate the LLMs' suggestions regarding their own work.
lol, lmao even. here is the invite and my reply
January 5, 2026 at 11:40 PM
The author's should ask ChatGPT to provide feedback on the risks of nonresponse bias in their design.
If you're trying to design-bias a study towards LLM-friendly findings, this is how you do it.
If you're trying to design-bias a study towards LLM-friendly findings, this is how you do it.
Reposted by Joe Bak-Coleman
I'm teaching Statistical Rethinking again starting Jan 2026. This time with live lectures, divided into Beginner and Experienced sections. Will be a lot more work for me, but I hope much better for students.
I will record lectures & all will be found at this link: github.com/rmcelreath/s...
I will record lectures & all will be found at this link: github.com/rmcelreath/s...
December 9, 2025 at 1:58 PM
I'm teaching Statistical Rethinking again starting Jan 2026. This time with live lectures, divided into Beginner and Experienced sections. Will be a lot more work for me, but I hope much better for students.
I will record lectures & all will be found at this link: github.com/rmcelreath/s...
I will record lectures & all will be found at this link: github.com/rmcelreath/s...
Reposted by Joe Bak-Coleman
who wants to see baby howlers? also look at that mom's face! she looks exhausted and exasperated because the babies don't stop moving for a second! unless they're asleep in ridiculous poses.
January 2, 2026 at 11:00 PM
who wants to see baby howlers? also look at that mom's face! she looks exhausted and exasperated because the babies don't stop moving for a second! unless they're asleep in ridiculous poses.
I’ve been in compliance
January 3, 2026 at 12:12 AM
I’ve been in compliance
What makes this remarkable to me is that we have many published papers relying on metas data, and they’ve demonstrated a willingness to alter data being externally evaluated.
Unlike Pruitt or Gino, we have no way of checking the integrity of data they’ve provided in the past.
Unlike Pruitt or Gino, we have no way of checking the integrity of data they’ve provided in the past.
We got Meta’s “general global playbook” for defeating advertiser verification regulations, which the company knows would reduce scams. It includes making scam ads “not findable” for regulators searching Meta’s ad library through targeted scrubbing.
www.reuters.com/investigatio...
www.reuters.com/investigatio...
Meta created ‘playbook’ to fend off pressure to crack down on scammers, documents show
As regulators pressure Meta to verify the identity of advertisers on Facebook and Instagram, the social media giant has drafted a “playbook” to stall them. A Reuters investigation examines its tactics...
www.reuters.com
January 2, 2026 at 5:43 PM
What makes this remarkable to me is that we have many published papers relying on metas data, and they’ve demonstrated a willingness to alter data being externally evaluated.
Unlike Pruitt or Gino, we have no way of checking the integrity of data they’ve provided in the past.
Unlike Pruitt or Gino, we have no way of checking the integrity of data they’ve provided in the past.
Must read reporting
We got Meta’s “general global playbook” for defeating advertiser verification regulations, which the company knows would reduce scams. It includes making scam ads “not findable” for regulators searching Meta’s ad library through targeted scrubbing.
www.reuters.com/investigatio...
www.reuters.com/investigatio...
Meta created ‘playbook’ to fend off pressure to crack down on scammers, documents show
As regulators pressure Meta to verify the identity of advertisers on Facebook and Instagram, the social media giant has drafted a “playbook” to stall them. A Reuters investigation examines its tactics...
www.reuters.com
December 31, 2025 at 11:07 PM
Must read reporting
New reporting from @jeffhorwitz.bsky.social shows meta actively altering shared data to produce rose-tinted results in searches of the as library.
Can we reasonably trust the data they share with academics?
www.reuters.com/investigatio...
Can we reasonably trust the data they share with academics?
www.reuters.com/investigatio...
Meta created ‘playbook’ to fend off pressure to crack down on scammers, documents show
As regulators pressure Meta to verify the identity of advertisers on Facebook and Instagram, the social media giant has drafted a “playbook” to stall them. A Reuters investigation examines its tactics...
www.reuters.com
December 31, 2025 at 9:17 PM
New reporting from @jeffhorwitz.bsky.social shows meta actively altering shared data to produce rose-tinted results in searches of the as library.
Can we reasonably trust the data they share with academics?
www.reuters.com/investigatio...
Can we reasonably trust the data they share with academics?
www.reuters.com/investigatio...
Reposted by Joe Bak-Coleman
Every week, Nature publishes yet another breathless puff piece about some AI startup, based only unpublished claims from the company and interviewing only those who work there.
How can the leading scientific journal publish piece after piece that would make Kevin Roose blush?
I think it's that...
How can the leading scientific journal publish piece after piece that would make Kevin Roose blush?
I think it's that...
December 30, 2025 at 10:08 PM
Every week, Nature publishes yet another breathless puff piece about some AI startup, based only unpublished claims from the company and interviewing only those who work there.
How can the leading scientific journal publish piece after piece that would make Kevin Roose blush?
I think it's that...
How can the leading scientific journal publish piece after piece that would make Kevin Roose blush?
I think it's that...
It's strange how rarely journalists covering this space can ask "Hey, but is it bullshit?". CEO's whose companies are over-leveraged, bubbly, and unprofitable promise a major breakthrough that will justify their fragile position and no one seems to *consider* they may be lying.
It's Theranos 2.0.
It's Theranos 2.0.
Gary, I hope you know that I really appreciate your views as a way to keep myself honest instead of falling into the easy traps of minimizing what AI can do.
Yet I find this kind of talk to strain credulity at best.
Yet I find this kind of talk to strain credulity at best.
Where Is All the A.I.-Driven Scientific Progress?
www.nytimes.com
December 30, 2025 at 5:44 PM
It's strange how rarely journalists covering this space can ask "Hey, but is it bullshit?". CEO's whose companies are over-leveraged, bubbly, and unprofitable promise a major breakthrough that will justify their fragile position and no one seems to *consider* they may be lying.
It's Theranos 2.0.
It's Theranos 2.0.
700 miles driven in a day is a bit much.
December 29, 2025 at 4:06 PM
700 miles driven in a day is a bit much.
*in expectation*
That’s not how GDP works. But it is how propaganda works.
December 24, 2025 at 2:05 PM
*in expectation*
It sticks out in this article that #openscience top brass quickly pivoted from critiquing “gold standard” science to embracing it.
Helping the very same nih director who is slashing science in the us, politicizing it, and shared conspiracy theories about me and my colleagues work.
Helping the very same nih director who is slashing science in the us, politicizing it, and shared conspiracy theories about me and my colleagues work.
December 23, 2025 at 1:45 PM
It sticks out in this article that #openscience top brass quickly pivoted from critiquing “gold standard” science to embracing it.
Helping the very same nih director who is slashing science in the us, politicizing it, and shared conspiracy theories about me and my colleagues work.
Helping the very same nih director who is slashing science in the us, politicizing it, and shared conspiracy theories about me and my colleagues work.
Reposted by Joe Bak-Coleman
December 21, 2025 at 6:28 PM
Reposted by Joe Bak-Coleman
Agree with the quoted statement by @richarddmorey.bsky.social and disagree with Gelman on that point of asymmetry. Much of what we call metascience today has gained popularity and momentum from scientists/policy makers/funders is by making major positive claims about how science works/doesn't work.
“I think there’s an argument to be made that much meta-scientific work is a kind of mirror image of the empirical work it critiques”
statmodeling.stat.columbia.edu/2025/12/21/i...
statmodeling.stat.columbia.edu/2025/12/21/i...
“I think there’s an argument to be made that much meta-scientific work is a kind of mirror image of the empirical work it critiques” | Statistical Modeling, Causal Inference, and Social Science
statmodeling.stat.columbia.edu
December 21, 2025 at 4:47 PM
Agree with the quoted statement by @richarddmorey.bsky.social and disagree with Gelman on that point of asymmetry. Much of what we call metascience today has gained popularity and momentum from scientists/policy makers/funders is by making major positive claims about how science works/doesn't work.
Reposted by Joe Bak-Coleman
This smells distinctly like collider bias and/or selection bias and/or regression to the mean... You simply can't select teen prodigies, and world class athletes rom databases, and go run regressions without serious consideration of the selection process!
"Most top achievers (Nobel laureates and world-class musicians, athletes, chess players) demonstrated lower performance than many peers during their early years. Across the highest adult performance, peak performance is negatively correlated with early performance" www.science.org/doi/10.1126/...
Recent discoveries on the acquisition of the highest levels of human performance
Scientists have long debated the origins of exceptional human achievements. This literature review summarizes recent evidence from multiple domains on the acquisition of world-class performance. We re...
www.science.org
December 20, 2025 at 3:48 PM
This smells distinctly like collider bias and/or selection bias and/or regression to the mean... You simply can't select teen prodigies, and world class athletes rom databases, and go run regressions without serious consideration of the selection process!
Reading the structured abstract alone should set off a half dozen statistical alarms in any scientists head.
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
Recent discoveries on the acquisition of the highest levels of human performance
Scientists have long debated the origins of exceptional human achievements. This literature review summarizes recent evidence from multiple domains on the acquisition of world-class performance. We re...
www.science.org
December 21, 2025 at 4:06 PM
Reposted by Joe Bak-Coleman
Updatedletter, I know go into soem emperical results as well: zenodo.org/records/1800...
The Apperent Negative Associations Between Early and Adult Performance in elite performers Arise from Colider Selection Bias and Baserate Neglect
Güllich et al. argue that among elite performers there is a negative association between early and adult performance, a pattern they link to distinct developmental causal mechanisms for early, and adu...
zenodo.org
December 21, 2025 at 4:04 PM
Reading the structured abstract alone should set off a half dozen statistical alarms in any scientists head.
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
Recent discoveries on the acquisition of the highest levels of human performance
Scientists have long debated the origins of exceptional human achievements. This literature review summarizes recent evidence from multiple domains on the acquisition of world-class performance. We re...
www.science.org
December 21, 2025 at 3:23 PM
Reading the structured abstract alone should set off a half dozen statistical alarms in any scientists head.
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
The fact that this found its way into Science makes me wonder how we drive home those stats 101 lessons and make them intuitive.
www.science.org/doi/10.1126/...
These claims about what AI can do truly need citations. Checking statistics isn’t routine work, nor is whether citations to the literature are sensible.
Whether statistics are appropriate and render reasonable inferences is inherently a question of human judgement.
Whether statistics are appropriate and render reasonable inferences is inherently a question of human judgement.
December 20, 2025 at 3:46 AM
These claims about what AI can do truly need citations. Checking statistics isn’t routine work, nor is whether citations to the literature are sensible.
Whether statistics are appropriate and render reasonable inferences is inherently a question of human judgement.
Whether statistics are appropriate and render reasonable inferences is inherently a question of human judgement.
Reposted by Joe Bak-Coleman
@jbakcoleman.bsky.social and I had some thoughts about this a while back.
AI, peer review and the human activity of science
When researchers cede their scientific judgement to machines, we lose something important.
www.nature.com
December 16, 2025 at 11:13 PM
@jbakcoleman.bsky.social and I had some thoughts about this a while back.
Gonna be rejecting any paper that draws any inference about people from this approach.
Did you know that from tomorrow, Qualtrics is offering synthetic panels (AI-generated participants)?
Follow me down a rabbit hole I'm calling "doing science is tough and I'm so busy, can't we just make up participants?"
Follow me down a rabbit hole I'm calling "doing science is tough and I'm so busy, can't we just make up participants?"
December 16, 2025 at 6:47 PM
Gonna be rejecting any paper that draws any inference about people from this approach.
This is pretty much how making coffee for my wife goes every morning.
HARD BOILED DETECTIVE: another cruel sunrise over the city of the damned. another corpse. another widow. another cup of black coffee to drag my sorry ass back to life- *sputtering*
ME: do you like it? it's an ethiopian washed process single origin
DETECTIVE:
ME: are you getting apricot notes
ME: do you like it? it's an ethiopian washed process single origin
DETECTIVE:
ME: are you getting apricot notes
December 16, 2025 at 5:32 PM
This is pretty much how making coffee for my wife goes every morning.
Reposted by Joe Bak-Coleman
at this point no one cares, but i was invited to talk about the high-rep retraction saga, and once again, i find myself perplexed at the exclusion of confirmatory results in any calculation in a study making a key claim about the inclusion of confirmatory studies making results highly replicable.
December 15, 2025 at 9:51 PM
at this point no one cares, but i was invited to talk about the high-rep retraction saga, and once again, i find myself perplexed at the exclusion of confirmatory results in any calculation in a study making a key claim about the inclusion of confirmatory studies making results highly replicable.
Going well over on the aixiv, guess the vibe code didn’t think to render latex.
December 14, 2025 at 10:25 PM
Going well over on the aixiv, guess the vibe code didn’t think to render latex.