However, the absolute differences between these model groups strongly depended on the measured concept, and we observed strong variance in performance among models of the same group.

We present results for over 50 metrics side-by-side to judge the opportunity costs of choosing one method over another. The results revealed strong variation across different groups of models. Overall, modern methods (transformers and generative AI) outperform the older, simpler ones.

Would be giving a talk on Data Quality Framework, online… Hopefully can see some of your faces

Subscribe here - groups.google.com/a/g-groups.w...

On the side of publishing, i wonder if that is domain-specific problem or a general issue across fields? What I feel like is that most subdomains are created to cater the need for understanding particularities… but perhaps maybe in less niche journals, we do need more overarching theories.

Certainly - quality has little to do with model / theoretical complexity. I guess what i am trying to get at is that generalisation of theories requires testings on edge cases to ensure robustness of the theories themselves, meaning that particularities are only the means, not the ends.

Accounting for complexity. Moreover, even in establishing statistical laws, we are still interested in cases where such laws do (not) apply and therefore we require peculiarities to validate the universality of general models.

Absolutely agree that we need to find regularities and formalism in media effects research.

But not sure how conducive it is to reduce (model) complexity - my intuition is that the current methods are simply bad at accounting for nuance, but methods from complex system and statmec are good at …