Indeed - I don’t know precisely how qed is built though and the extent to which it is just LLMs. This how Oded described it and it is trained with lots of human feedback www.qedscience.com/blog/authors...

yes. I worry the "let AI do the science for us now" means we end up trying to milk more and more out of the same data. we're already looking for our keys in the lamplight, and AI encourages this more with an even narrower beam... But I think an author-facing tool to improve papers is different

That was my impression to!

yep - that's exactly my feeling. maybe it won't work but we should see

that is very true

yep - but only by trying it would we know

have you tried qed?

I agree. my original feeling was machine analysis would be good for identifying things like known contaminated cell lines and all sorts of standard checks but not for peer review. That's why I'm intrigued by this new tool and the positive response it has got from a lot of people known to be cynics

part of the issue is that journals are having real trouble finding enough human reviews and papers can be in limbo for months as dozens decline (I saw this at JCS). Meanwhile preprint authors often complain no-one in the community is giving them the feedback they'd hoped for.

the key to me is to establish what parts of manuscript evaluation can be automated to allow people to focus on the parts that can't.

that's literally the implementation at biorxiv. authors can send a paper there if they want to get feedback - just like they could to any of >300 journals and various independent human review services. just another option. I too have heard mostly good things, but some have been more meh

true - the question is when for a given task it's unreliability becomes comparable to humans'

well the journal system is buckling because journals can't get human reviewers. meanwhile preprint authors complain they don't get enough comments from humans on papers. so part of the problem is humans have taken themselves out..
.

yep - one would hope that AI can save time and augment/complement human effort but always the worry that the lazy use it

yes - the irony had occurred to me too. (Ibtw evidence based not assumption given people claim this is like AI slop papers). And the biorxiv-qed partnership is to help authors (many of whom complain they get no human feedback) get analyses of their papers so it is hardly about discouraging reading.

oh I'm certainly not accusing you of that. I think there are lots of reasons to be cautious. But these things exist and authors want to use them - so, as with journals, we are trying to help by providing interoperability

plenty of AI initiatives working with arxiv info.arxiv.org/labs/showcas...

As for "reflexive loathing", I'm not saying all of the concerns are (some are reasonable, and I have my own ) but the fact many are clearly damning this without having read what it is actually doing (e.g. assuming it's about writing papers, assuming it's all papers) is evident it's reflexive 2/2