then you can reply with

thanks for mentioning our preprint!

we're currently in revision; keen to hear any devastating critiques so the paper is as useful as it can be :)

i LOVED getting over it! will check out :)

good timing!

Also check out this paper by Jonathan de Quidt, Johannes Haushofer, and Christopher Roth deriving bounds for demand effects in the dictator game (here, we directly replicate their "weak" demand cue in a different sample) www.aeaweb.org/articles?id=...

yes, thanks for your interest! the preprint is here: osf.io/preprints/ps...

(i never know whether the algorithm penalizes threads with a link in the first post)

haha, well, at least 4% of people say shape-shifting lizards control the govt.

Also, some great work by Seetahul and Greitemeyer suggests that participants are more likely to react when they think studies will counteract their interests (journals.sagepub.com/doi/full/10....)

One thing we can't rule out: a mixture of demand compliance AND reactance in the same person (i.e., feeling pulled in both directions). But I'm not sure what kind of experiment could test this easily. A straightforward within-subjects design could be subject to concerns of "meta demand."

We also don't see a very sharp difference in the standard deviations in both demand conditions (which we'd expect if we have reacters and compliers). Distributions look pretty similar.

Good point! We address this in study 3 (p. 27), where we fit a mixture model testing for latent classes of compliers and reacters. No latent class exhibited significant evidence of a shift from zero, either in the compliance or reactance direction. The subsample which trended closest was <5% of Ps

Thrilled to work with Lucas Woodley, @rcalcott.bsky.social, & @fierycushman.bsky.social on this project! (Also, glad that many of our causal estimates seem to be unbiased by demand.) Lots more in the paper if you’re interested: osf.io/g6xhf_v1

In short: do you need to worry about experimenter demand ruining *your* online study? Based on our evidence, probably not.

That's good news for the field! As we argue, demand effects appear, at least in their simplest form, to be more phantom than menace (7/8)

Then we measured participants' dictator game behavior, moral vignette judgments, and change in ingroup attitudes after an intervention (we used an inert subliminal priming intervention for measurement purposes).

Control and demand conditions were statistically indistinguishable! (6/8)

To answer this, we used obvious ("We hypothesize...") and subtle demand manipulations ("These images are designed to make you feel more warmth toward the average [conservative/liberal]")

In each case we verified participants correctly understood study hypotheses. (5/8)

...and demand effects are most often observed with small student samples or very heavy-handed cues ("You will help us if you...")

But modern psychology experiments use experienced online samples and standardized paradigms. Is demand a realistic concern in this setting? (4/8)

Some background: meta-analysis (@nicholascoles.bsky.social, Morgan Wyatt, & Michael C. Frank) and prior large-scale studies using economic games (@jondequidt.bsky.social, @johanneshaushofer.com, & Christopher Roth) find small though inconsistent demand effects... (3/8)