Tymon Słoczyński
@tymonsloczynski.bsky.social
Associate Professor of Economics at Brandeis University, interested in microeconometrics, applied econometrics, causal inference, and economic history.
Personal website: https://tslocz.github.io/
Personal website: https://tslocz.github.io/
Application (unilateral divorce laws → female suicide, 1964–1996): TWFE ≈ –0.6 (ns). ATT estimators ≈ –10.2 (p=0.001) and –5.5 (p=0.138). Internal validity of TWFE ≈ 0 (uniform), rising to ~40–60% with estimated effects. Not very representative here. 13/13
October 23, 2025 at 3:27 PM
Application (unilateral divorce laws → female suicide, 1964–1996): TWFE ≈ –0.6 (ns). ATT estimators ≈ –10.2 (p=0.001) and –5.5 (p=0.138). Internal validity of TWFE ≈ 0 (uniform), rising to ~40–60% with estimated effects. Not very representative here. 13/13
Example (3 time periods): internal validity is highest when two-thirds of units are treated in the last period – that's when TWFE weights become uniform. 12/13
October 23, 2025 at 3:27 PM
Example (3 time periods): internal validity is highest when two-thirds of units are treated in the last period – that's when TWFE weights become uniform. 12/13
For TWFE DiD: the weights can be negative, so in general no uniform causal interpretation. If group-time effects are constant over time, weights ≥ 0, and our measure tells how representative TWFE is of treated/all units. 11/13
October 23, 2025 at 3:27 PM
For TWFE DiD: the weights can be negative, so in general no uniform causal interpretation. If group-time effects are constant over time, weights ≥ 0, and our measure tells how representative TWFE is of treated/all units. 11/13
For IV/2SLS: except in special cases, no obvious ranking in terms of internal validity between "interacted" and "non-interacted" specifications studied in recent papers. 10/13
October 23, 2025 at 3:27 PM
For IV/2SLS: except in special cases, no obvious ranking in terms of internal validity between "interacted" and "non-interacted" specifications studied in recent papers. 10/13
A few implications for familiar designs: Under unconfoundedness, OLS estimand has internal validity ≤ 4·P(D=1)·P(D=0).
E.g., if 10% of units are treated, OLS can represent ≤ 36% of the population.
9/13
E.g., if 10% of units are treated, OLS can represent ≤ 36% of the population.
9/13
October 23, 2025 at 3:27 PM
A few implications for familiar designs: Under unconfoundedness, OLS estimand has internal validity ≤ 4·P(D=1)·P(D=0).
E.g., if 10% of units are treated, OLS can represent ≤ 36% of the population.
9/13
E.g., if 10% of units are treated, OLS can represent ≤ 36% of the population.
9/13
When the CATE function is known: if the estimand lies in the convex hull of CATE values, a causal representation exists. The max subpopulation size then follows from a simple linear program with a closed-form solution. 8/13
October 23, 2025 at 3:27 PM
When the CATE function is known: if the estimand lies in the convex hull of CATE values, a causal representation exists. The max subpopulation size then follows from a simple linear program with a closed-form solution. 8/13
Practitioner use: you can compute this diagnostic at the design stage from treatment or instrument propensities – no outcome data needed. Then, use it to understand your implicit subpopulation better and/or form simple bounds on the target ATE. 7/13
October 23, 2025 at 3:27 PM
Practitioner use: you can compute this diagnostic at the design stage from treatment or instrument propensities – no outcome data needed. Then, use it to understand your implicit subpopulation better and/or form simple bounds on the target ATE. 7/13
Key result (no assumptions on how effects vary): the estimand has a clear causal interpretation if and only if its weights a(X) are nonnegative. Then max internal validity = E[a(X) ∣ W₀=1]/sup a(X). Negative weights ⇒ no uniform causal meaning. 6/13
October 23, 2025 at 3:27 PM
Key result (no assumptions on how effects vary): the estimand has a clear causal interpretation if and only if its weights a(X) are nonnegative. Then max internal validity = E[a(X) ∣ W₀=1]/sup a(X). Negative weights ⇒ no uniform causal meaning. 6/13
We call P(W*=1 ∣ W₀=1) the internal validity of the estimand – how much of your intended population it really represents. P(W*=1) is representativeness – how much of the full population it covers. 5/13
October 23, 2025 at 3:27 PM
We call P(W*=1 ∣ W₀=1) the internal validity of the estimand – how much of your intended population it really represents. P(W*=1) is representativeness – how much of the full population it covers. 5/13
Notation:
– Y(1),Y(0): potential outcomes.
– D: binary treatment.
– X: covariates.
– a(X): weights defining the estimand.
– W₀: population of interest (e.g., all, treated, compliers).
– W*: subgroup for which the estimand = avg. effect. 4/13
– Y(1),Y(0): potential outcomes.
– D: binary treatment.
– X: covariates.
– a(X): weights defining the estimand.
– W₀: population of interest (e.g., all, treated, compliers).
– W*: subgroup for which the estimand = avg. effect. 4/13
October 23, 2025 at 3:27 PM
Notation:
– Y(1),Y(0): potential outcomes.
– D: binary treatment.
– X: covariates.
– a(X): weights defining the estimand.
– W₀: population of interest (e.g., all, treated, compliers).
– W*: subgroup for which the estimand = avg. effect. 4/13
– Y(1),Y(0): potential outcomes.
– D: binary treatment.
– X: covariates.
– a(X): weights defining the estimand.
– W₀: population of interest (e.g., all, treated, compliers).
– W*: subgroup for which the estimand = avg. effect. 4/13
Set-up: Many familiar estimands (OLS, 2SLS, TWFE) are weighted averages of heterogeneous treatment effects.
We ask: (i) When does such an estimand equal the avg. effect for some group? (ii) How large can that group be?
3/13
We ask: (i) When does such an estimand equal the avg. effect for some group? (ii) How large can that group be?
3/13
October 23, 2025 at 3:27 PM
Set-up: Many familiar estimands (OLS, 2SLS, TWFE) are weighted averages of heterogeneous treatment effects.
We ask: (i) When does such an estimand equal the avg. effect for some group? (ii) How large can that group be?
3/13
We ask: (i) When does such an estimand equal the avg. effect for some group? (ii) How large can that group be?
3/13
Paper: arxiv.org/abs/2404.14603
2/13
2/13
Quantifying the Internal Validity of Weighted Estimands
In this paper we study a class of weighted estimands, which we define as parameters that can be expressed as weighted averages of the underlying heterogeneous treatment effects. The popular ordinary least squares (OLS), two-stage least squares (2SLS), and two-way fixed effects (TWFE) estimands are all special cases within our framework. Our focus is on answering two questions concerning weighted estimands. First, under what conditions can they be interpreted as the average treatment effect for some (possibly latent) subpopulation? Second, when these conditions are satisfied, what is the upper bound on the size of that subpopulation, either in absolute terms or relative to a target population of interest? We argue that this upper bound provides a valuable diagnostic for empirical research. When a given weighted estimand corresponds to the average treatment effect for a small subset of the population of interest, we say its internal validity is low. Our paper develops practical tools to quantify the internal validity of weighted estimands. We also apply these tools to revisit a prominent study of the effects of unilateral divorce laws on female suicide.
arxiv.org
October 23, 2025 at 3:27 PM
Paper: arxiv.org/abs/2404.14603
2/13
2/13