🌍We hope this sparks further discussion on research practices in our field! See @thejop.bsky.social: www.journals.uchicago.edu/doi/10.1086/....

🛠️ To improve the reliability of applied natural/quasi-experimental work, we provide a checklist. Key steps include pre-registering measurement choices, exploring a range of operationalizations, and testing for what we call vertical and horizontal observable implications.

📈 This leads to vast, mostly invisible, researcher degrees of freedom. These matter for applied research: we show for the case of extreme weather that a broad picture from many operationalizations indicates robust null effects, while singular measures could support positive or negative relations.

🔍 We identify a wide range of potential/applied operationalizations in 3 exemplary literatures drawing on external shocks: extreme weather, immigration, commodity prizes. This is due to two features relevant to many other natural/quasi-experiments: lacking theory + reliance on threshold decisions.