But you can already just go to their website.

The issue is bioRxiv implicitly endorsing the service and telling authors "hey, try this."

I had a thread the other day on how qed is a good facsimile of your avg peer reviewer. The avg peer reviewer isn't great...

On this point, I think humans are already bad at this. The founder of plate tectonics failed to get his PhD because his panel didn't believe him, etc...

But it's true... I don't think the way we train AI does equip it to break the infobank we gave it.

Mind if I ask for your take @carlbergstrom.com ? Would be curious where your thoughts on AI review are present and future.

I can't help but feel AI logic regresses to the mean as it's trained with good+bad papers and reviews.

Experts can discard bad info. AI reliably incorporates it.

bsky.app/profile/hans...

The lesson from this thread isn't that qed is as good as a peer reviewer. It's that qed is as good as your average*** peer reviewer.

Peer review is broken. The average peer reviewer is bad. We didn't need or want to scale that up!

Would be curious on your thoughts re: something I'm working on. Will DM!

And likely be just as annoying

Yes!

In the field, it's well-established that gene is an ideal readout, and the authors say as such: "geneX is a readout of I-pathway." But the chatbot has seen umpteen reviews that say "you should test additional genes" so that's what it says.

2/2

I actually don't think that's what's happening based on the kinds of gaps and its suggestions.

Ex. "You say this, but measure just one gene as a readout for a pathway. You should measure additional genes to ensure your result is pathway level and not gene-specific."

1/2

I don't know so much about burden tests. How would burden tests shake out on this situation? Where would this example fit in the lexicon of specificity and importance? And is this countervailing conundrum part of the difference between GWAS and Burden rank differences? 4/4

This ex. sticks with me as a note on how GWAS relies on the tested pop. having sufficient variant rates in important genes, AND those variants don't interfere with each other. 3/4

Paper here:

The case I'm thinking of, 1% of individuals had stop & some 14% had the SNP. Both cause sensitivity to infection (confirmed, proven independently). But the 1% that have the stop have the 'good' variant of the SNP, and in GWAS including stop individuals changes SNP Pvals by 2 orders of magnitude 2/4

Thanks for the thread! This'll be our next journal club paper for sure. Congrats!

Can I ask: GWAS can lose sensitivity for important variants if two variants in a gene exist, but have countervailing effects. eg, premature stop & important SNP. Stop rare, SNP more common. 1/4

📌

Definitely agree on resisting current LLMs. In my thread I'm generous to the idea of incorporating them within the current* publishing infrastructure in place of reviewer 2.

But only because the current system's bad & we should spend less time on the bad part because it's ineffective time spend.

It's funny that I just tried out qed and put out a mammoth thread on my experience:

bsky.app/profile/hans...

Though to be honest, my first impressions are rather poor. The 3 preview comments on the first paper I uploaded just pointed out how the AI has poor reading comprehension and lacks the ability to judge the data within the figures. The 3 comments I saw were each semantic and not meaningful.