Tony Albano
@thetaminusb.bsky.social
UC Davis School of Education - testing, measurement, quant methods
Equating happens, in some fashion, basically every year with every state test. This isn't news at all. And ironically, it's *not using equating* that would represent a change in standard because you might happen to pass only because you got an easier test.
August 17, 2025 at 11:37 PM
Equating happens, in some fashion, basically every year with every state test. This isn't news at all. And ironically, it's *not using equating* that would represent a change in standard because you might happen to pass only because you got an easier test.
NY state didn't "lower the bar" and there wasn't any "tweaking" or "test-tinkering" as the article claims. From what I can tell (no sources are given), they increased scores in some grades and subjects to account for tests that were slightly more difficult in 2025 vs 2024. That's just equating.
August 17, 2025 at 11:37 PM
NY state didn't "lower the bar" and there wasn't any "tweaking" or "test-tinkering" as the article claims. From what I can tell (no sources are given), they increased scores in some grades and subjects to account for tests that were slightly more difficult in 2025 vs 2024. That's just equating.
R1 faults the author for responding precisely to actual critiques and not addressing “possible critiques.”
March 31, 2025 at 9:50 PM
R1 faults the author for responding precisely to actual critiques and not addressing “possible critiques.”
I confessed that I’ve never used them for coding, or really for anything, and the class was shocked 😬 Maybe I’ve just optimized regular old online search when I have a problem? Or maybe I’m missing out on some big gains?
March 26, 2025 at 9:03 PM
I confessed that I’ve never used them for coding, or really for anything, and the class was shocked 😬 Maybe I’ve just optimized regular old online search when I have a problem? Or maybe I’m missing out on some big gains?
We went over the pros and cons - LLMs are a great resource but use with discretion and always work through to reinforce understanding.
March 26, 2025 at 9:03 PM
We went over the pros and cons - LLMs are a great resource but use with discretion and always work through to reinforce understanding.
I think it’s fair for testing companies to preserve data they collect outside operational testing, for research purposes. To protect their IP. States testing data should be public though.
March 15, 2025 at 3:02 PM
I think it’s fair for testing companies to preserve data they collect outside operational testing, for research purposes. To protect their IP. States testing data should be public though.
I think the benefits of open-access data are worth the negligible risks if we deidentify and have large N. Most educational testing data should be public.
March 15, 2025 at 2:42 PM
I think the benefits of open-access data are worth the negligible risks if we deidentify and have large N. Most educational testing data should be public.
Link to the open-access article: www.thelancet.com/journals/lan...
School phone policies and their association with mental wellbeing, phone use, and social media use (SMART Schools): a cross-sectional observational study
There is no evidence that restrictive school policies are associated with overall
phone and social media use or better mental wellbeing in adolescents. The findings
do not provide evidence to support ...
www.thelancet.com
February 7, 2025 at 10:03 PM
Link to the open-access article: www.thelancet.com/journals/lan...
DIF is a last resort for detecting anomalies in scores due to problems in the test that prior intense scrutiny might have missed. In that sense it does its job pretty well.
January 24, 2025 at 6:15 PM
DIF is a last resort for detecting anomalies in scores due to problems in the test that prior intense scrutiny might have missed. In that sense it does its job pretty well.
Circularity is a fair critique and would be a blow to test design were it not for all the other stuff we do, including qualitative research, to create unbiased test content. The book doesn’t talk about any of that.
January 24, 2025 at 6:15 PM
Circularity is a fair critique and would be a blow to test design were it not for all the other stuff we do, including qualitative research, to create unbiased test content. The book doesn’t talk about any of that.
Every DIF analysis I’ve read assumes, without a solid check, that there are enough unbiased items to anchor the detection of biased ones. We should take criterion measures more seriously, especially with our highest stakes tests.
January 24, 2025 at 6:15 PM
Every DIF analysis I’ve read assumes, without a solid check, that there are enough unbiased items to anchor the detection of biased ones. We should take criterion measures more seriously, especially with our highest stakes tests.
DIF analysis is circular. Ideally, we’d compare our test to a trusted criterion. If groups differ on one but not the other, it suggests bias. But I’ve never seen DIF run with an external measure. We bootstrap, controlling for overall group differences using the test itself as we analyze item scores.
January 24, 2025 at 6:15 PM
DIF analysis is circular. Ideally, we’d compare our test to a trusted criterion. If groups differ on one but not the other, it suggests bias. But I’ve never seen DIF run with an external measure. We bootstrap, controlling for overall group differences using the test itself as we analyze item scores.
If you 10x access to stuff that is not learning, you get lower achievement, especially for students with less support and fewer buffers.
January 24, 2025 at 5:38 PM
If you 10x access to stuff that is not learning, you get lower achievement, especially for students with less support and fewer buffers.
“Determining the drivers of student achievement trends is not simple. Most conventional explanations for student performance trends struggle to fully account for the patterns identified in this report.”
There’s no simple answer but we all know it’s social and other online media right?
There’s no simple answer but we all know it’s social and other online media right?
January 24, 2025 at 5:38 PM
“Determining the drivers of student achievement trends is not simple. Most conventional explanations for student performance trends struggle to fully account for the patterns identified in this report.”
There’s no simple answer but we all know it’s social and other online media right?
There’s no simple answer but we all know it’s social and other online media right?
Do Standardized Tests Benefit from Inequality? – theta minus b
thetaminusb.com
January 3, 2025 at 6:08 AM
The book gives three references here, none of which support the claims that test scores must be bell shaped to be valid, or that state tests in particular are all scaled to have a normal shape. I think it's just a misunderstanding. Or maybe I'm missing something.
January 3, 2025 at 6:08 AM
The book gives three references here, none of which support the claims that test scores must be bell shaped to be valid, or that state tests in particular are all scaled to have a normal shape. I think it's just a misunderstanding. Or maybe I'm missing something.
But state accountability testing isn't just for normative comparison. It's mainly for comparison to performance standards. Norms can also be applied, but, since they aren't designed for comparison among test takers, state tests aren't tied to inequality in results.
January 3, 2025 at 6:08 AM
But state accountability testing isn't just for normative comparison. It's mainly for comparison to performance standards. Norms can also be applied, but, since they aren't designed for comparison among test takers, state tests aren't tied to inequality in results.
I think I know what this is getting at. Norm referenced tests (e.g., selection, prediction) are optimized when there is variability in scores - if everyone does well or poorly, scores are bunched up and it's harder to make comparisons. Technically, norm referencing does benefit from inequality.
January 3, 2025 at 6:08 AM
I think I know what this is getting at. Norm referenced tests (e.g., selection, prediction) are optimized when there is variability in scores - if everyone does well or poorly, scores are bunched up and it's harder to make comparisons. Technically, norm referencing does benefit from inequality.
While slayers of testing often oversimplify and misconstrue their enemy, I was surprised to see this basic distortion of test design - "all standardized tests are designed to produce what's called a bell curve..."
January 3, 2025 at 6:08 AM
While slayers of testing often oversimplify and misconstrue their enemy, I was surprised to see this basic distortion of test design - "all standardized tests are designed to produce what's called a bell curve..."
The book is dramatic at times - it sets the stage with testing as a monster to be slayed - but I've been looking for a good summary of the Marxist, critical-theory, anti-testing perspective, and this seems to fit.
January 3, 2025 at 6:08 AM
The book is dramatic at times - it sets the stage with testing as a monster to be slayed - but I've been looking for a good summary of the Marxist, critical-theory, anti-testing perspective, and this seems to fit.