The Unjournal (Unjournal.org)
banner
unjournal.bsky.social
The Unjournal (Unjournal.org)
@unjournal.bsky.social
Researchers, practitioners, & open science advocates building a better system for research evaluation. Nonprofit. We commission public evaluation & rating of hosted work. To make rigorous research more impactful, & impactful research more rigorous.
Reposted by The Unjournal (Unjournal.org)
We *are* doing this.

There is the @unjournal.bsky.social. There is Peer Community In @peercommunityin.bsky.social. There are dozens of newly-founded diamond OA (free to read, free to publish) journals being born, eg Experimental Philosophy @xphilosopher.bsky.social.
November 14, 2025 at 11:20 AM
Unjournal.org is planning to work with:
https://bit.ly/49lWRVJ (Consider pledging?)

Plans:
- Integrate some Unjournal’s Pivotal Question content (cultured meat, plant-based alternatives)
- Help operationalize & vet tournament questions
Metaculus' First Animal-Focused Forecasting Tournament
Project summary This project aims to run the first animal-focused forecasting tournament on the Metaculus platform. The goal is to generate forecasts on decision-relevant questions that will help both funders and animal-aligned organisations make more informed decisions about their funding allocation and strategy. Questions will be crowdsourced from the collective animal advocacy community and vetted appropriately to ensure the questions’ relevancy to decision-making in the community. The tournament contains a prize pool, which will be split amongst the best forecasters. What are this project's goals? How will you achieve them? The primary goals of this project are threefold: To enable funders in the animal welfare/rights space to prioritize animal-aligned interventions more effectively. To raise awareness amongst animal-aligned organisations about forecasting, to enable them to make better-informed decisions about their strategies and future direction. We also want forecasters outside the animal advocacy community to weigh in on the forecasts, diversifying the set of forecasters for the questions in the tournament. #1 and #2 will primarily be achieved by advertising the tournament in various animal advocacy communities (e.g. Hive, Sentient Futures) and partnering with a media body like Sentient Media. I’m hoping to achieve #3 by the presence of a prize pool in the tournament, so if forecasters weren’t intrinsically motivated to forecast these questions, at least the money to be won would be a good motivator. How will this funding be used? The funding will be used to create a prize pool for the Metaculus tournament. If funding exceeds $10,000, a small portion of the amount above the $10,000 will also be used to cover my efforts in filtering questions, operationalising them to a Metaculus-friendly format, advertising the tournament, and following up with decision makers in the animal advocacy community to gauge whether they have found the forecasts useful. The minimum funding is set such that the prize pool will hopefully be viable to incentivize a sufficient number of forecasters to predict. More money raised will produce a larger prize pool, greater participation incentive, and more attention on the tournament, and will also enable more outreach to spread awareness. Who is on your team? What's your track record on similar projects? It’s primarily me on the team. Jeffrey Maxim, a forecasting enthusiast and blogger from the Hive community, and Ryan Beck, the Forecasting Programs Coordinator at Metaculus are playing supporting roles. The team at The Unjournal will also be collaborating with me on selecting and refining the final set of questions. I have not personally worked on something similar, but Ryan Beck is experienced in shaping and operationalising Metaculus tournaments, including many of Metaculus’ active and past tournaments. What are the most likely causes and outcomes if this project fails? Questions evolve to become non-decision-relevant over time. Funders and animal organisations are unsure how to translate forecasts into concrete actions/decisions Aggregate forecasts on the tournament questions are completely unreflective of the actual outcome, leading decision makers to potentially make the wrong decisions How much money have you raised in the last 12 months, and from where? None.
bit.ly
November 15, 2025 at 10:40 PM
We are delighted to welcome Paolo Crosetto to our advisory board at The Unjournal. Paolo is a full-time senior researcher (Directeur de Recherche, DR) at INRAE. He works in Grenoble, France, within the Research Unit GAEL – Grenoble Applied Economics Laboratory.
November 13, 2025 at 5:03 PM
How long do Unjournal evaluators take to do their evaluations? For our academic stream, about 8 hours on average, with substantial dispersion.

(Scatterplot: imputed data from 40 quantifiable responses, not including applied stream nor those who used the PubPub form.)
November 12, 2025 at 11:48 PM
Looking for examples of high-quality peer reviews of impactful research in economics & social science? See Unjournal's "commended evaluations": https://bit.ly/4qJo0rX

Recently commended work by 3 David's (Reiley, Hugh-Jones, & Manheim), Estere Seinkmane, Eleanor Tsai, & 2 anonymous heroes.
Overview · Prize-winning & commended evaluations
bit.ly
November 7, 2025 at 3:08 PM
Researchers: Has your paper been evaluated by The Unjournal (see unjournal.pubpub.org)?

We want your feedback on the experience, the quality of the evaluations, and how this impacted your work.

https://bit.ly/4qKtYIZ

1/10 get a $100 prize.
Author Survey Form
Please note that all questions are optional. By completing this form, you will enter a draw with a chance of winning $100. Every 1 out of 10 respondents will receive the compensation.
bit.ly
November 6, 2025 at 4:36 PM
We're inviting academic journals to recognize The Unjournal's public evaluation packages in their editorial process.

- Help w/ editorial triage, review quality, & turnaround
- Support transparency, open-access.

Details: https://bit.ly/3JEee9J

Feedback & questions welcome!
Traditional journals can use Unjournal evaluations | The Unjournal: project and communication space
bit.ly
November 5, 2025 at 10:48 PM
All of this is a small fraction of the billions paid to for-profit publishers for "Article Processing Charges", of course.

opusproject.eu/openscience-...

(Maybe $50 million per year in APCs for physics and astronomy the most active areas for SciPost)
October 30, 2025 at 8:15 PM
If you're among the 2000+ authors citing research evaluated by Unjournal.org, go to unjournal.pubpub.org to learn how commissioned experts rated & assessed that research.

Top 'citers of unjournal-evaluated research': https://bit.ly/3WRHc8W include Esther Duflo, Julian Jamison, & Berk Özler
October 22, 2025 at 4:16 PM
Congrats to Mokyr, Aghion & Howitt for the Nobel memorial prizes in economics sciences!

And, you guessed it: This is a blatant promo for one of our first & strongest evaluation packages, of "Artificial Intelligence & Economic Growth” by Aghion et al.
“Artificial Intelligence and Economic Growth”: Evaluation summary & metrics
Summary, metrics and ratings, and Manager's comments on Evaluation of “Artificial Intelligence and Economic Growth” by Aghion et al.
bit.ly
October 13, 2025 at 4:49 PM
Evaluation Summary and Metrics: "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"
Evaluation Summary and Metrics: "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"
Evaluation Summary and Metrics: ""Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"" for The Unjournal.
bit.ly
October 10, 2025 at 7:44 PM
Reposted by The Unjournal (Unjournal.org)
Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->
October 8, 2025 at 9:34 PM
Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->
October 8, 2025 at 9:34 PM
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys" for The Unjournal.
bit.ly
October 8, 2025 at 8:40 PM
I love what #SciPost does & their narrative.
Much agrees w/ Unjournal.org practices.

Some differences (Us/Them)
1. Quant. ratings vs accept/reject
2. Publish all evaluations vs. only if ‘accepted’
3. Fields & applied focus
4. We pay evaluators
5. We don’t 'publish' papers, just evaluations
October 3, 2025 at 6:25 PM
Climate change, environmental econ., biodiversity: Of the 50+ papers Unjournal.org commissioned for evaluation and assessment, ten focus on these areas, offering insight into methods & policy.

See our 'collection': https://bit.ly/4pKF0NT -->
September 29, 2025 at 11:34 AM
Unjournal.pubpub.org: 50+ detailed packages evaluating research w/ potential for impact.

But who funded the *research*? GiveWell @poverty-action.bsky.social, @gatesfoundation.bsky.social, ERC, NIH, @wellcometrust.bsky.social, & ~40 others

See our dashboard: tinyurl.com/22k6ashr

-->
September 27, 2025 at 11:52 AM
Benabou et al ~measure “are people are utilitarian or deontological” via lab experiments (incl. "real stakes trolley problem").

→ 20-44% non-consequentialist choices, but unstable preferences

2 evaluators: interested but skeptical of design & interpretation.
Evaluation Summary and Metrics: "Ends versus Means: Kantians, Utilitarians, and Moral Decisions"
Evaluation Summary and Metrics: "Ends versus Means: Kantians, Utilitarians, and Moral Decisions" for The Unjournal.
bit.ly
September 24, 2025 at 2:56 PM
https://bit.ly/3Vvz4KN

Love Brian Nosek's response to my Q on "fear of negative public evaluations" @ Unjournal & Lifecyclejournal

"I certainly understand the fear. I shared it myself until I had the experience of realizing that's how people advance their careers." -->
BITSS AM 2025: The Evolution of Research Transparency - Brian Nosek
Brian Nosek, Executive Director of the Center for Open Science, delivered the keynote presentation at the BITSS 2025 Annual Meeting. His…
www.youtube.com
September 22, 2025 at 10:04 AM
Why cite Unjournal.org evals in your research?

1. Strengthen/temper cited claims
2. Justify methods/revisions
3. Credit insights
4. Support meta-analysis

How? Easy citation interfaces, individual DOIs for each eval & response

Short (uslick) explainer:
How and why to cite The Unjournal's evaluations in your research and practice
Quick discussion of why and how to cite Unjournal evaluations when doing research, whitepapers, etc. Quick walk-through of how to do so.…
youtu.be
September 20, 2025 at 3:46 PM
Did you know The Unjournal has a researcher and evaluator pool with:

309 prioritized members, 170 highly prioritized (rated 65%+)

110 with PhD degrees

147 Economists

Dashboard: https://bit.ly/4gxFZws

How to get involved & use our work: https://bit.ly/3ImqTxu
September 20, 2025 at 12:45 PM
Evaluation Summary and Metrics: "Maternal cash transfers for gender equity and child development: Experimental evidence from India"
Evaluation Summary and Metrics: "Maternal cash transfers for gender equity and child development: Experimental evidence from India"
Evaluation Summary and Metrics: "Maternal cash transfers for gender equity and child development: Experimental evidence from India" for The Unjournal.
bit.ly
September 16, 2025 at 11:32 AM
Evaluation Summary and Metrics: "A systematic review and meta analysis of social safety nets, women’s economic achievements and agency"
Evaluation Summary and Metrics: "A systematic review and meta analysis of social safety nets, women’s economic achievements and agency"
Evaluation Summary and Metrics: "A systematic review and meta analysis of social safety nets, women’s economic achievements and agency" for The Unjournal.
bit.ly
September 16, 2025 at 11:10 AM
Rethink Priorities 2022 research: “Forecasts estimate limited cultured meat production through 2050“.

Unjournal 2025 evaluators: The forecasts and the report were too pessimistic.
Manheim: cultured meat is likely to be commercially viable soon.

https://bit.ly/4nhyIDX

#CellAg #AnimalWelfare

-->
Evaluation Summary and Metrics: "Forecasts estimate limited cultured meat production through 2050", Applied Stream
Evaluation Summary and Metrics: "Forecasts estimate limited cultured meat production through 2050" for The Unjournal (Applied stream)
bit.ly
September 11, 2025 at 12:28 PM
Evaluation Summary and Metrics: "Forecasts estimate limited cultured meat production through 2050", Applied Stream
Evaluation Summary and Metrics: "Forecasts estimate limited cultured meat production through 2050", Applied Stream
Evaluation Summary and Metrics: "Forecasts estimate limited cultured meat production through 2050" for The Unjournal (Applied stream)
bit.ly
September 9, 2025 at 10:44 AM