Take a look at the full paper:
- science.org/doi/10.1126/...

I started this project 5 years ago. Paper with my former students Han Zhuang and Lizhen Liang

Disclaimer: I am the founder of
ReviewerZero AI (www.reviewerzero.ai),
where we help with research integrity issues.

A few takeaways:
- the AI of course still makes mistakes, but we believe those mistakes are worth it
- predictions should be part of a triage system with experts. we make our predictions "interpretable", aligning with DOAJ guidelines
- But AI for integrity is here to stay

Applied to journals that haven't been vetted before , we predict more than 1000 new potentially questionable journals.
They have collectively published over 500K articles, cited millions of times, and acknowledged major funders in US, China, and Japan

We tried several methods, including simple regression, random forest and deep learning.
- Bibliometric features alone: PRC AUC ≈0.64
- Combined model: PRC AUC ≈0.79

Agreement with DOAJ guidelines and expert reviewers was strong, though false positives remain.

We trained models on ~15,000 journals labeled by DOAJ (12,869 legitimate vs 2,536 removed). Features included:
- Website content (editorial boards, policies)
- Website design (HTML structure, screenshots)
- Bibliometrics (citations, author metrics)

that makes sense! 🧠

NotebookLM podcast version of the paper. Pretty good! notebooklm.google.com/notebook/d75... (need a Google account)