Megan Stevenson
@meganstevenson.bsky.social
economist & legal scholar studying criminal justice issues.
professor of law
professor of economics (by courtesy)
UVA
https://sites.google.com/view/megan-stevenson/home
professor of law
professor of economics (by courtesy)
UVA
https://sites.google.com/view/megan-stevenson/home
Declination rates are near zero in Philadelphia and zero in Harris County (since the police call into the DA’s office for approval before making an arrest) so you could also work with court data sets from those counties and be confident you’re not missing much
November 10, 2025 at 10:55 PM
Declination rates are near zero in Philadelphia and zero in Harris County (since the police call into the DA’s office for approval before making an arrest) so you could also work with court data sets from those counties and be confident you’re not missing much
www.jstor.org/stable/10.10...
I think then you would just need police data linked to court data since a prosecutor’s decision to charge would result in a charge in the court records. The data used in the paper above should do the trick.
I think then you would just need police data linked to court data since a prosecutor’s decision to charge would result in a charge in the court records. The data used in the paper above should do the trick.
Racial Disparity in Federal Criminal Sentences on JSTOR
M. Marit Rehavi, Sonja B. Starr, Racial Disparity in Federal Criminal Sentences, Journal of Political Economy, Vol. 122, No. 6 (December 2014), pp. 1320-1354
www.jstor.org
November 10, 2025 at 10:55 PM
www.jstor.org/stable/10.10...
I think then you would just need police data linked to court data since a prosecutor’s decision to charge would result in a charge in the court records. The data used in the paper above should do the trick.
I think then you would just need police data linked to court data since a prosecutor’s decision to charge would result in a charge in the court records. The data used in the paper above should do the trick.
If you want something vaguely nationally representative, but a bit outdated, the SCPS data has that for a number of large jurisdictions from 2009
November 10, 2025 at 8:03 PM
If you want something vaguely nationally representative, but a bit outdated, the SCPS data has that for a number of large jurisdictions from 2009
Not totally sure what you are asking? Most court data will have that except for arrests that did not lead to a charge. If court data is what you want, there are lots of data sets out there and I’m happy to share some with you if you want.
November 10, 2025 at 8:02 PM
Not totally sure what you are asking? Most court data will have that except for arrests that did not lead to a charge. If court data is what you want, there are lots of data sets out there and I’m happy to share some with you if you want.
But just look at the literature. It is chock full of completely implausible claims.
Agreed that judge IV is not worse than many other IV settings, both are BAD.
A number of studies show IV estimates ~10 times larger than OLS even when the theory predicts OLS to be biased away from zero.
Agreed that judge IV is not worse than many other IV settings, both are BAD.
A number of studies show IV estimates ~10 times larger than OLS even when the theory predicts OLS to be biased away from zero.
November 7, 2025 at 2:09 PM
But just look at the literature. It is chock full of completely implausible claims.
Agreed that judge IV is not worse than many other IV settings, both are BAD.
A number of studies show IV estimates ~10 times larger than OLS even when the theory predicts OLS to be biased away from zero.
Agreed that judge IV is not worse than many other IV settings, both are BAD.
A number of studies show IV estimates ~10 times larger than OLS even when the theory predicts OLS to be biased away from zero.
I’m saying that power tends to be extremely low with this research design and the bias induced by significance filtering goes up as bias goes down. There’s also lots of degrees of freedom which enables cherry picking as well as potential monotonicity and exclusion violations.
November 7, 2025 at 2:02 PM
I’m saying that power tends to be extremely low with this research design and the bias induced by significance filtering goes up as bias goes down. There’s also lots of degrees of freedom which enables cherry picking as well as potential monotonicity and exclusion violations.
When the literature systematically produces massive, unrealistic estimates, I feel like we should approach the research design as if there are flashing red lights and blaring sirens warning danger, not a "keep calm and ujive on."
November 7, 2025 at 1:54 PM
When the literature systematically produces massive, unrealistic estimates, I feel like we should approach the research design as if there are flashing red lights and blaring sirens warning danger, not a "keep calm and ujive on."
It’s not an equal concern with all research designs. It is a particular concern with low power research designs and judge IV tends to be very low power. I honestly think it has been a bad thing for our field to embrace it as the majority of estimates it produces are extremely biased.
November 7, 2025 at 1:52 PM
It’s not an equal concern with all research designs. It is a particular concern with low power research designs and judge IV tends to be very low power. I honestly think it has been a bad thing for our field to embrace it as the majority of estimates it produces are extremely biased.
In practice, I think judge IV is a hugely biased and unreliable estimator. It is typically an order of magnitude larger than OLS estimates, even when theory would predict that OLS is biased away from zero.
November 7, 2025 at 1:50 PM
In practice, I think judge IV is a hugely biased and unreliable estimator. It is typically an order of magnitude larger than OLS estimates, even when theory would predict that OLS is biased away from zero.
In practice I think this is really important. Judge IV estimates tend to be an order of magnitude larger than OLS even when theory would suggest OLS is biased away from zero.
November 7, 2025 at 1:48 PM
In practice I think this is really important. Judge IV estimates tend to be an order of magnitude larger than OLS even when theory would suggest OLS is biased away from zero.
Strong disagree. The paper does not consider issues of publication bias/specification search. In the presence of any sort of significance filtering (which we know is rampant) judge IV tends to be an extremely biased estimator since only the extreme estimates are statistically significant.
November 7, 2025 at 1:46 PM
Strong disagree. The paper does not consider issues of publication bias/specification search. In the presence of any sort of significance filtering (which we know is rampant) judge IV tends to be an extremely biased estimator since only the extreme estimates are statistically significant.
Paper link: papers.ssrn.com/sol3/papers....
The Importance of Base-Rates in Differential Impact: A Bail Reform Case-Study
Researchers commonly seek to understand whether a policy has differential impact on different subgroups, yet the choice of differential impact metric can have a
papers.ssrn.com
October 24, 2025 at 3:31 PM
Paper link: papers.ssrn.com/sol3/papers....
With multiple reasonable methods of calculating racially disparate impact the researcher needs to think carefully about what they want to convey.
And probably report multiple metrics for full transparency. Fin. 14/14
And probably report multiple metrics for full transparency. Fin. 14/14
October 24, 2025 at 3:31 PM
With multiple reasonable methods of calculating racially disparate impact the researcher needs to think carefully about what they want to convey.
And probably report multiple metrics for full transparency. Fin. 14/14
And probably report multiple metrics for full transparency. Fin. 14/14
We look at racial disparities in impact for a Philly bail reform. Even though Black defendants benefited most in an absolute sense, White defendants were disproportionately selected to benefit (had higher BRAI).
Sounds complicated? That’s the point. 13/
Sounds complicated? That’s the point. 13/
October 24, 2025 at 3:31 PM
We look at racial disparities in impact for a Philly bail reform. Even though Black defendants benefited most in an absolute sense, White defendants were disproportionately selected to benefit (had higher BRAI).
Sounds complicated? That’s the point. 13/
Sounds complicated? That’s the point. 13/
In our setting we use it to show how characteristics of beneficiaries (i.e. compliers) differ from the pool of potential beneficiaries (i.e. compliers + never takers). It’s a cool method but probably too much to explain here. Below is a little teaser graph! 12/
October 24, 2025 at 3:31 PM
In our setting we use it to show how characteristics of beneficiaries (i.e. compliers) differ from the pool of potential beneficiaries (i.e. compliers + never takers). It’s a cool method but probably too much to explain here. Below is a little teaser graph! 12/
The paper also has a more technical bit showing how to calculate the base-rate adjusted impact in a difference-in-differences design. To do so, we adapt the complier analysis method shown in Jager et al (2019) for the IV-DD setting. 11/
October 24, 2025 at 3:31 PM
The paper also has a more technical bit showing how to calculate the base-rate adjusted impact in a difference-in-differences design. To do so, we adapt the complier analysis method shown in Jager et al (2019) for the IV-DD setting. 11/
All three metrics are valid, but answer different questions and yield very different answers!!
So proceed with *caution* when interpreting racial disparities in impact, and be sure to discuss how different metrics can yield different answers. 10/
So proceed with *caution* when interpreting racial disparities in impact, and be sure to discuss how different metrics can yield different answers. 10/
October 24, 2025 at 3:31 PM
All three metrics are valid, but answer different questions and yield very different answers!!
So proceed with *caution* when interpreting racial disparities in impact, and be sure to discuss how different metrics can yield different answers. 10/
So proceed with *caution* when interpreting racial disparities in impact, and be sure to discuss how different metrics can yield different answers. 10/
In other words, beneficiaries were not chosen in a manner orthogonal to race. White defendants were chosen to benefit at a *much* higher rate than Black defendants. This is the base-rate adjusted impact measure. 9/
October 24, 2025 at 3:31 PM
In other words, beneficiaries were not chosen in a manner orthogonal to race. White defendants were chosen to benefit at a *much* higher rate than Black defendants. This is the base-rate adjusted impact measure. 9/
With a 10 p.p. treatment effect, 100% of the White potential beneficiaries benefited from the reform, whereas only 14% (10/70) of Black potential beneficiaries *actually* benefited. We refer to this as the base-rate adjusted impact measure or BRAI. 8/
October 24, 2025 at 3:31 PM
With a 10 p.p. treatment effect, 100% of the White potential beneficiaries benefited from the reform, whereas only 14% (10/70) of Black potential beneficiaries *actually* benefited. We refer to this as the base-rate adjusted impact measure or BRAI. 8/
In this example, for every 100 Black defendants, 70 of them would have had to pay cash bail absent the reform (i.e. are potential beneficiaries). For White defendants, only 10 of them would have had to pay cash bail absent the reform. 7/
October 24, 2025 at 3:31 PM
In this example, for every 100 Black defendants, 70 of them would have had to pay cash bail absent the reform (i.e. are potential beneficiaries). For White defendants, only 10 of them would have had to pay cash bail absent the reform. 7/