This was a massive effort spearheaded by @sarahcen.bsky.social , and in collaboration with @aifi.bsky.social , Hedi Driss, Charlotte Park, @dataspen.bsky.social , and @aleksmadry.bsky.social.

Happy to discuss further with anyone who's interested in learning more! (6/6)

(iii) interrogate the models’ beliefs about candidates’ attributes, and (iv) reveal the models’ implicit predictions of the election outcome.

We’re making the dataset publicly available and invite researchers to run their own analyses on it: huggingface.co/datasets/sar.... (5/6)

Using this dataset, we perform 4 analyses of the dataset that (i) study the longitudinal variation of model behavior during election season, (ii) illustrate the sensitivity of election-related responses to demographic steering, (4/6)

Our design systematically varies question content and format, resulting in a rich dataset that enables analyses of the models’ behavior over time (e.g., across model updates), sensitivity to steering, responsiveness to instructions, election-related knowledge and “beliefs.” (3/6)

We conducted a large-scale, longitudinal study of 12 models, queried using a structured survey with over 12,000 questions on a near-daily cadence from July through November 2024. (2/6)