Yeah, we had some small experiments embedded in the lucid sample. No differences between prof and non-prof. It is not in the main paper (sm materials) because we only have results for one of the three samples.

Thank YOU for the opportunity to work together on this, Jonathan!

Yeah.. absolutely. We make our efforts to control for this by imposing a long break between repeated attempts (30min if I’m not mistaken). But substantive results don’t change much with longer break like 1h or even 24h

We use URLs from platforms we can identify a unique survey. So not all platforms we have in the paper and def not proprietary platforms. So think for example about qualtrics. In this case, a survey taker can try to take it more than once with the url, switching browsers, switching profiles, etc..

Me too! As someone who does most of my research outside of the U.S. I’m also very curious about this phenomenon in other markets! Hopefully our paper can help in the measurement challenges of survey taking for future research!

Yeah… that’s a great point we tried to be as thoughtful as possible! We provide a sensitivity analysis for this case, and show how our prevalence estimates would change. We discuss this limitation in detail in the paper!

Try to! We really do not know if they were able to complete the survey page they were visiting

Thank you, Amanda!

ohh very cool! Can you share the paper? I would love to read it!

The paper is co-authored with Bernhard Von Clemm, @ericka.bric.digital
@jonathannagler.bsky.social @magdalenawojciesza.bsky.social

This is also one of the projects I started at @csmapnyu.org ---- thanks to the entire lab involved!

The paper can be found here: www.cambridge.org/core/journal...

Huge caveat: our data comes from a period before the widespread use of Generative AI.

How professionals are using these tools, and how easy access to GenAI models might affect the quality of survey research, remains an open question for future research.

Our conclusion is that survey professionalism is widespread on online panels, but these do not, by and large, distort inferences, making us cautiously optimistic about research using these online samples!

But one concerning pattern emerges:

Professionals are considerably more likely to attempt to take the same survey repeatedly.

For example, roughly 85 % of Lucid professionals tried to take at least one survey more than once

c) When looking at the stability of responses over time, we find no evidence of professionals being less attentive or more likely to change their responses across survey waves.

We take this as evidence that they answer questions at least as attentively as non-professionals.

Our results show:

a) No robust demographic or political differences between professionals and non-professionals.

b)Professionals are more likely to speed and attempt repeats, but those are easy to screen out in surveys.

And..

Next, we ask: Do professionals risk survey quality and inference from survey research? We test this in three dimensions:

a) demographic & political differences;
b) low-effort behavior (speeding, straightlining),
c) survey stability across waves

Prevalence varies sharply across platforms. Our most conservative estimate shows:

- 1.7 % of Facebook respondents are professionals
- 7.6 % on YouGov
- 34.7 % on Lucid

Professionalism is a real phenomenon, but it varies widely across samples!

We analyze 3,886 respondents who donated their browsing histories (~96M web visits). We measure actual survey-taking, rather than relying on self-reports, as much of prior work on this topic does.

We focus on three outcomes: prevalence, data quality, and survey repetition.

I use the YouTube API, but with someone’s else wrapper. To actually show how to query an api, I go with nyt. It is simple and easy to use.