And I, too, would think I was beating a dead horse if it weren't for the fact that top journals keep publishing simple lag-1 CLPMs with no controls (meta-analytic CLPMs are still CLPMs): www.apa.org/pubs/journal...

Reposting this thread about another recent preprint that discusses some of the reasons why: bsky.app/profile/rich...

Rates of significant effects are also arguably too high with the RI-CLPM. Individual effects are about half as likely to be significant with the RI-CLPM as with the CLPM, but you still get at least one significant lagged effect in 61% of models.

Anyway, we welcome any thoughts or suggestions you have about the paper! [15/15]

Also: state variance is not just measurement error, so using latent variables with the CLPM or RI-CLPM won't always fix this (though it helps). Even with multiple-item Big Five scores modeled using latent traits, the same pattern emerged. [14/x]

Our analyses also suggest that once you account for state variance, stabilities of variables included in these panel studies are extremely high: Median 1-year stability = .90. This suggests that there is often very little change occurring that could be accounted for by a lagged effect [13/x].

Our analyses suggest that the CLPM and RI-CLPM would be the most appropriate model for just 4% of variables we examined; the other 96% were divided equally between ARTS and STARTS [12/x]

What type of model can account for these patterns? Models that include a state component (like the STARTS or even the ARTS, which drops the stable trait). You can reproduce actual patterns of stability even without assuming the existence of a stable trait (this plot is for the ARTS) [11/x]

It turns out it's very common. Here are the plots of stability over increasingly long lags for about 400 variables from a large panel study. For anything less than almost perfect short-term stability, stability coefficients should reach an asymptote long before 22 years; very few do [10/x]

One pattern it can't handle is when short-term stability is moderate, medium-term stability is just slightly lower than long-term stability, and there is no clear asymptote with increasingly long lags (as is true with life satisfaction and health). How commons is this pattern? [9/x]

Why doesn't the RI-CLPM just estimate a lower asymptote (corresponding to less stable trait variance)? It turns out that the RI-CLPM is quite limited in the types of patterns of stability with which it is compatible (won't go into details here, but paper does). [8/x]

This gets even worse with more waves of data. In our experience, this is a pretty common pattern: If you have more than 4 or 5 waves, the RI-CLPM overestimates long-term stability (and often underestimates medium-term stability). This misfit can also lead to bias in estimates [7/x]

The RI-CLPM does better, but even here, there is a hint of misspecification (see how stability coefficients continue to decline even after the RI-CLPM predicts an asymptote) [6/x]

Here's a plot of actual stability coefficients and those implied by a fitted CLPM with life satisfaction and health. It's really bad...CLPM dramatically underestimates stability. This misspecification introduces bias, which can be severe [5/x]

There are already lots of good papers comparing these models from a causal inference perspective, but we focus more on typical patterns of stability in real data and how well they match with the patterns of stability implied by these models. Why is this useful? Consider the CLPM. [4/x]

TL/DR: More reasons why the standard lag-1 CLPM is really bad, but also some important concerns about models that assume the existence of a stable trait (like the RI-CLPM). Our analyses suggest that we should be paying more attention to state variance when modeling these effects [3/x]

We try to do three new things. 1. Focus explicitly on model misfit and its implications for bias. 2. Conduct simulations showing how specific forms of misfit affect bias. 3. Analyze the longitudinal structure of over 400 variables to see what type of misfit is likely for specific models. [2/x]

Will get right on that (checks to see whether @syeducation.bsky.social has something I can use as a template that I can pass on to @jnfrltackett.bsky.social )