Item Response Warehouse
@itemrespwarehouse.bsky.social
The IRW is a collection of harmonized datasets available for psychometrics research.
https://itemresponsewarehouse.org/
https://itemresponsewarehouse.org/
Lots of item-level heterogeneity in treatment sensitivity across a wide-range of RCTs (consistent with the earlier findings from www.tandfonline.com/doi/abs/10.1... that focused on education-based RCTs).
June 23, 2025 at 2:16 AM
Lots of item-level heterogeneity in treatment sensitivity across a wide-range of RCTs (consistent with the earlier findings from www.tandfonline.com/doi/abs/10.1... that focused on education-based RCTs).
That leader being @sachaepskamp.bsky.social !
March 3, 2025 at 2:38 PM
That leader being @sachaepskamp.bsky.social !
See also additional description in section 4.4 of the updated IRW preprint. osf.io/preprints/ps...
OSF
osf.io
December 16, 2024 at 4:14 PM
See also additional description in section 4.4 of the updated IRW preprint. osf.io/preprints/ps...
7/ Many of these data are available for reuse on the IRW! Next week we'll share code to help you start replicating these analyses!
December 12, 2024 at 11:02 PM
7/ Many of these data are available for reuse on the IRW! Next week we'll share code to help you start replicating these analyses!
6/ Understanding whether an empirical study is one versus the other may help us refine our understanding of what is really happening in the treatment.
December 12, 2024 at 11:02 PM
6/ Understanding whether an empirical study is one versus the other may help us refine our understanding of what is really happening in the treatment.
5/ This paper shows that many treatments increase the response averages (ie a conventional treatment effect) while others serve to tighten connections between items (while not changing the overall level of item response).
December 12, 2024 at 11:02 PM
5/ This paper shows that many treatments increase the response averages (ie a conventional treatment effect) while others serve to tighten connections between items (while not changing the overall level of item response).
4/ A separate paper (osf.io/preprints/ps...) pushes further to consider treatment effects from these studies from the perspective of network psychometrics.
OSF
osf.io
December 12, 2024 at 11:02 PM
4/ A separate paper (osf.io/preprints/ps...) pushes further to consider treatment effects from these studies from the perspective of network psychometrics.
3/ It goes further and shows that we may misunderstand uncertainty if we do not account for item-level variation and demonstrates identification challenges associated with the estimation of treatment-by-covariate interaction effects (a big deal given interest in heterogeneous treatment effects).
December 12, 2024 at 11:02 PM
3/ It goes further and shows that we may misunderstand uncertainty if we do not account for item-level variation and demonstrates identification challenges associated with the estimation of treatment-by-covariate interaction effects (a big deal given interest in heterogeneous treatment effects).
2/ This study, along with previous work (www.tandfonline.com/doi/abs/10.1...), shows that item-level variation in treatment effects is a real problem in empirical data!
Heterogeneity of Item-Treatment Interactions Masks Complexity and Generalizability in Randomized Controlled Trials
Researchers compare aggregate test data across treatment and control group in randomized controlled trials to determine intervention impact on student achievement. We show that item-level data and ...
www.tandfonline.com
December 12, 2024 at 11:02 PM
2/ This study, along with previous work (www.tandfonline.com/doi/abs/10.1...), shows that item-level variation in treatment effects is a real problem in empirical data!
See website for more details!
datapages.github.io/irw/
datapages.github.io/irw/
Item Response Warehouse
Open: The data are licensed for reformatting and public reuse. Their provenance is documented and code used to transform original data to meet the IRW data standard is available.Harmonized: The data a...
datapages.github.io
December 10, 2024 at 5:40 PM
See website for more details!
datapages.github.io/irw/
datapages.github.io/irw/
You can access these datasets straight from R or Python using API calls. Paperwork (citation & license) on the provenance of all inclusions available too.
December 10, 2024 at 5:39 PM
You can access these datasets straight from R or Python using API calls. Paperwork (citation & license) on the provenance of all inclusions available too.