Thanks to all the commentary authors for your contributions, and thanks especially to the editors at Meta-psychology especially for allowing us to test out this format for our publication. It was a long road, but the end result is in my opinion terrific!

Quantifying replication value as a combination of citation count and sample size is our first stab at solving a very complex problem. Such early attempts benefit enormously from being critiqued right away. We were really lucky to receive several excellent commentaries from many experts in the field.

This paper forms one of Meta-Psychology’s Special Topics. Eight commentaries have been published alongside the paper which criticizes and extends the ideas presented within. We have written a response to these commentaries here: open.lnu.se/index.php/me...

Indeed, I'm looking forward!

True, and maybe that should be emphasized in the blog post. Will consider adding a section at the end of the post. Any sources you'd recommend I'd cite in a section like this?

To be clear, the point of this post is not to say only experiments support causal inference and correlational research never can. Quite the opposite in fact. If that was your takeaway, I may need to add some language to clarify.

The motivation for picking the example in this post was simply that I wanted a thought experiment that a 1st year undergraduate in most areas of social and health science could wrap their head around without any additional reading.

I actually use the history of smoking~cancer research as my running example when introducing our bachelor students to research design and experimental vs observational research. It's a terrific example, albeit on a tragic subject.

Yes, one of several practical problems. You have to assume a perfect implementation for the hypothesis diagram to be true without caveats. Still, coming from a field where none of this is made explicit in most textbooks, I think understanding the unrealistically perfect case can be helpful.

Given that I wanted to cap the post at ~1000 words a lot of nuance will obviously be lost. Still, I think this post provides a useful preface to a proper causal inference introduction like Pearl's primer book.

Thanks for the reference, will check this out! I absolutely agree, and I don't think I say anything of the sort. However, I wanted to write a post that in very simple terms lays out the causal graphical logic underlying the textbook statement "you have to use experiments to make causal claims".

Later on the authors recommend abductive inference. I agree with the recommendation. However, abduction that leads us to a causal conclusion is just causal inference by another name. Saying that causal inference is not allowed is not sage advice. Better to emphasize that causal inference is hard.

I understand that sensible causal inference based on (cross-sectional) correlations can be hard. Very hard. But that does not mean our goal should not be causal inference. If a network modeling approach invalidates any causal inference, I question the usefulness of such a modeling approach.

Absolutely, feel free!

I mean, randomized control is usually mentioned in any explanation, but why is randomization to begin with? Why does randomization and manipulation give experiments a leg up on observational research when it comes to causal inference? This blog post is an attempt to explain why, briefly.

The idea that if you're a poor researcher you should automatically be assigned to teach instead makes about as much sense as the idea that if you're a poor teacher you should automatically be assigned to do research instead.

"Those who cannot do, teach" is, I think, another sacred cow of academia. "Those who cannot teach, do" strikes me as equally true. A good teacher seems more valuable to the academy than a good researcher, on average.