Kyle Marquardt
LightNews LightNews
banner
kailmarkvart.bsky.social
Kyle Marquardt
@kailmarkvart.bsky.social
930 followers 900 following 120 posts
Professor, @sampol.bsky.social; Project Manager, @vdeminstitute.bsky.social; Iowan; klmarquardt.com
Posts Replies Media Videos
kailmarkvart.bsky.social
Turkey, 26.10-06.11.25
Lion carving on On Gözlü Bridge, Diyarbakir Yoros Castle, Istanbul Mardin panorama Sunset in Mardin
November 9, 2025 at 8:47 PM
kailmarkvart.bsky.social
Faroe Islands, late September 2025
View of Kunoy from Klakkur Black sheep eating grass near Kirkjubøur View from Slættaratindur Rainbow over Nólsoy
October 14, 2025 at 7:08 PM
kailmarkvart.bsky.social
Sidenote: Elsewhere colleagues (incl @acrowinghen.bsky.social , @medzihorsky.bsky.social, @danpemstein.com, @chknutsen.bsky.social, @silindberg.bsky.social) and I find little evidence that bad-vibes among V-Dem’s country experts drive the recent global democratic decline.
doi.org/10.1017/S104...
Abstract: During the past decade, analyses drawing on several democracy measures have shown a global trend of democratic retrenchment. While these democracy measures use radically different methodologies, most partially or fully rely on subjective judgments to produce estimates of the level of democracy within states. Such projects continuously grapple with balancing conceptual coverage with the potential for bias (Munck and Verkuilen 2002; Przeworski et al. 2000). Little and Meng (L&M) (2023) reintroduce this debate, arguing that “objective” measures of democracy show little evidence of recent global democratic backsliding. By extension, they posit that time-varying expert bias drives the appearance of democratic retrenchment in measures that incorporate expert judgments. In this article, we engage with (1) broader debates on democracy measurement and democratic backsliding, and (2) L&M’s specific data and conclusions.
August 26, 2025 at 10:10 PM
kailmarkvart.bsky.social
Figure showing predicted relationship between perceptions of trends in democracy and V–Dem coding of liberal democracy indicators by type of perception and level of optimism in dataset. Relationship is clearly strongest for perceptions of local democracy trends (c), but even in this case has massive uncertainty.
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
Equally importantly, we examined the degree to which perceptions of democracy trends correlate with actual V-Dem coding, i.e., the degree to which experts changed their scores for specific variables in their main country of expertise 2012-2023. 12/17
Table showing results from linear regression analyses of V–Dem coding trends on perceptions of political trends. Results are strongest for perceptions of local democracy trends (1-3), and for sets of variables more closely linked to democracy (1-3, 5-6).
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
We find that, although the prime about global democracy trends does slightly decrease positivity about local trends, this effect is not significant, providing little evidence of a causal effect of global on local perceptions. 11/17
Table showing t-test results of priming experiment with a null result for perceptions of local democracy trends.
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
First, V-Dem experts are pessimistic (or realistic, depending on your perspective): 88% believe the level of global democracy has declined in the past decade. Expert perceptions of global and local (country-specific) trends also correlate. 7/17
Table showing results from linear regression analyses of perceptions of local trends in democracy on perceptions of global trends. There is a robust relationship between these two variables.
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
...leading them to erroneously lower their ratings of democratic institutions in their countries of expertise. These biased ratings in turn result in the negative trends in global democracy in expert-coded datasets, reinforcing the pessimistic narrative about democracy. 4/17 doi.org/10.1017/S104...
Abstract: Despite the general narrative that the world is in a period of democratic decline, there have been surprisingly few empirical studies that assess whether this is systematically true. Most existing studies of global backsliding are based largely if not entirely on subjective indicators that rely on expert coder judgment. Our study surveys objective indicators of democracy (e.g., incumbent performance in elections) and finds little evidence of global democratic decline during the past decade. To explain the discrepancy in trends between expert-coded and objective indicators, we consider the role of coder bias and leaders strategically using more subtle undemocratic action. Although we cannot rule out the possibility that the world is becoming less democratic exclusively in ways that require subjective judgment to detect, this claim is not justified by existing evidence.
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
Excited to introduce a new working paper with @danpemstein.com, @chknutsen.bsky.social and other colleagues not on Bluesky! We use experimental and observational data to investigate the degree to which general biases affect expert perceptions of specific cases. 🧵 1/17 v-dem.net/media/public...
Title: Global Trends and Expert Perceptions: Pessimism and the Assessment of Democratic Backsliding. Abstract There are longstanding concerns about biases in expert-coded data. Here we attempt to causally identify the effect of these biases, specifically evaluating the claim that pessimism about global democracy affects expert coding of democracy locally in specific countries. Observational analyses of data from an original survey of experts demonstrate that perceptions of global and local democratic trends are indeed correlated. However, an embedded experiment yields no evidence of a causal effect of perceptions of global democracy trends on perceptions of local trends. Moreover, we find little evidence that pessimism about global trends substantially affects actual coding behavior with regard to specific countries. Together, these results alleviate some concerns about expert biases both in widely-used measures of democracy and more broadly.
August 26, 2025 at 10:05 PM
kailmarkvart.bsky.social
Very much would love to hear what you find! I checked the V-Dem data and it doesn't look like there's any clear trend in the average across countries since the 1990s (though a trend weighted by population shows a gradual decline since the 1990s). v-dem.net/data_analysi...
V-Dem world average with equal weights for countries shows increase in late 1980s and no consistent trend afterward V-Dem population-weighted world average shows increase in outcome acceptance in the late eighties followed by gradual (but substantial) decline
February 27, 2025 at 9:48 PM
kailmarkvart.bsky.social
Super interesting project! In light of recent discussions would be fascinating to see how this indicator compares to the V-Dem indicator of a similar concept, as well as intercoder reliability stats on the coding of the new indicator.
V-Dem Codebook Entry for variable “election losers accept results” Little and Meng coding categories
February 27, 2025 at 11:08 AM
kailmarkvart.bsky.social
Many krai opportunities indeed!
Soviet Atlas of the Peoples of the World, showing distribution of nationalities in the USA.
February 21, 2025 at 6:01 PM
kailmarkvart.bsky.social
🏆🚨🏆 Reminder 🏆🚨🏆

Please nominate doctoral dissertations for the @apsa.bsky.social Gabriel A. Almond Award, which honors the best doctoral dissertation in the field of comparative politics!

Deadline: February 12, 2025.

Submit nominations here: apsa.secure-platform.com/a/organizati...
Nominate doctoral dissertations for the Gabriel A. Almond Award. The award honors the best doctoral dissertation in the field of comparative politics. The deadline is February 12, 2025. Eligibility: Dissertations must have been successfully defended within the previous two calendar years (dissertations for the 2025 award must be defended in 2023 or 2024). APSA will accept only one nomination for the Almond Award per school or political science department. Nominations from non-PhD departments and institutions are welcome if the nominee is currently employed there. Self-nominations are accepted. Nominees do not have to be members of APSA, affiliated with an institution in the United States, or an American citizen in order to be considered for an award. The award is presented at the APSA Annual Meeting and carries a cash prize of $750.  The award was created in recognition of Dr. Gabriel Almond’s contributions to the discipline, profession, and Association.  He was a long-time faculty member at Stanford University and former APSA President (1966).  Almond’s scholarly work contributed directly to the development of theory in comparative politics and brought together work on the developing areas and Western Europe that prevented splintering into an array of disparate areas studies.  
January 7, 2025 at 12:23 AM
kailmarkvart.bsky.social
Onward to 2025.
Afternoon sunset in Bergen
December 31, 2024 at 12:30 PM
kailmarkvart.bsky.social
Also, you may be wondering "What about LLMs?" So did our reviewers! Though we ended up cutting this from the final draft of the article, here is a brief outline of our thoughts about the use of LLMs to replace traditional coders of #poliskydata.
Recent scholarship suggests that large language models (LLMs) can perform many of the tasks of trained coders and crowdworkers in the social sciences. While we do not test the substitutability of LLM output for either coders or experts in this article, we agree that it is plausible that LLMs could perform well on the sort of tasks where we found crowdworkers were most substitutable for coders. However, even in this context issues of information availability will be a critical issue: the bias of most LLMs toward materials in commonly-spoken languages will limit their ability to gather historical data for many world regions; their reliance on publicly available data also limits their access to more academic texts that may contain this information. With regard to expert-coded data, we believe the substitutability of LLMs is more dubious. In most cases, expert-coded output does not have a single “correct” answer, but rather a distribution of plausible responses. A primary goal of expert coding is therefore to aggregate over the scores of multiple experts with different evaluations to generate both a point estimate for a latent concept and estimates of uncertainty over these values. While LLMs can synthesize data to provide point estimates of latent concepts, assessing the uncertainty and reliability associated with these estimates is not a straightforward endeavor. Given both the sensitivity of LLMs to prompt phrasing and their tendency to unpredictably hallucinate sources and data (Linegar et al. 2023), neither of these concerns are trivial. Finally, as with coder-coded data, the accuracy of LLM estimates will likely be correlated with data availability; even more than with coder-coded data the plausible lack of diverse views and biases in LLM training data (Bender et al. 2021) could result in seriously flawed output. Indeed, LLM estimates may be systematically biased in cases where the most readily available data are wrong.
December 5, 2024 at 3:03 PM
kailmarkvart.bsky.social
Our conclusion lays out the overall implications of these results well.
We find little evidence that crowds can substitute for experts on IRS tasks. Our typology provides a possible explanation: most crowdworkers do not have the background, support or incentives to gather and analyse the types of data that experts do. IRS tasks are difficult to break into small chunks, and therefore it may not be surprising that crowdworkers perform poorly in our study. Nevertheless, since essentially every published study on crowdsourcing in political science reports on a success, it is valuable to highlight the method’s limitations. However, we do find evidence for the substitutability of crowdworkers for coders in some tasks, though the boundaries of this substitutability are somewhat blurry. Although our results demonstrate that crowdworkers can substitute for coders in some simple information retrieval tasks, the difficulty of the task has a strong negative relationship with their substitutability.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
This figure shows the probability that a crowdworker would provide a correct response to trained-coder questions with different characteristics. Crowdworkers are substantially more likely to provide correct responses for more recent years and simpler questions.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
This result holds true in other expert-coded data contexts. This figure shows the degree to which different factors can make crowds more substitutable for expert-coded data, where lower values on the horizontal axis mean more substitutability.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
This is what happens when you ask a bunch of crowdworkers to use the same scale to code a subset of these years in Argentina. Trends are more difficult to discern.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
For example, this graphic shows the codings experts provided for trends in freedom from political killings in Argentina, 1916-2015. Each dot represents a coding, different colors different experts, with a smoothing line per expert. (Higher scores = more freedom, less killings).
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
In other recent work, colleagues (including @chknutsen.bsky.social, @acrowinghen.bsky.social, @medzihorsky.bsky.social, and @silindberg.bsky.social) and I have discussed the advantages and disadvantages of using experts vs trained coders to code #poliskydata. doi.org/10.1017/S104...
During the past decade, analyses drawing on several democracy measures have shown a global trend of democratic retrenchment. While these democracy measures use radically different methodologies, most partially or fully rely on subjective judgments to produce estimates of the level of democracy within states. Such projects continuously grapple with balancing conceptual coverage with the potential for bias (Munck and Verkuilen 2002; Przeworski et al. 2000). Little and Meng (L&M) (2023) reintroduce this debate, arguing that “objective” measures of democracy show little evidence of recent global democratic backsliding.1 By extension, they posit that time-varying expert bias drives the appearance of democratic retrenchment in measures that incorporate expert judgments. In this article, we engage with (1) broader debates on democracy measurement and democratic backsliding, and (2) L&M’s specific data and conclusions.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
Thrilled to have this #openaccess article, co-authored with seven colleagues including @danpemstein.com and @silindberg.bsky.social, published in IPSR! Shockingly long🧵follows. doi.org/10.1177/0192...
Political scientists increasingly use crowdworkers to produce data, predominantly in the context of coding researcher-curated text or to retrieve simple data from the internet. In this article, we provide a theoretical and empirical basis for understanding when crowdworkers can provide data of sufficient quality to substitute for other types of coders. First, we introduce a typology of data-producing actors – experts, trained coders and crowds – and hypothesize factors that affect the substitutability of crowdworkers. We then examine how crowdworkers perform across coding tasks that vary along multiple dimensions of difficulty: information verifiability, availability and complexity. The results provide scope conditions bounding the substitutability of crowdworkers in political science applications. Although crowds can substitute for trained coders in the context of relatively simple information retrieval tasks, there is little evidence that crowdworkers can substitute for experts, whose tasks require both information retrieval and data synthesis.
December 5, 2024 at 9:51 AM
kailmarkvart.bsky.social
Among other things, that means you can now examine trends in the characteristics of political elites in terms of gender and their association with their countries' socialist regimes...
Graphic showing percent of political elites who had active party membership during the socialist regime in six post-communist countries. Graphic showing share of women among political elites in six post-communist countries.
November 22, 2024 at 2:42 PM
kailmarkvart.bsky.social
Data from a small-scale Levada telephone survey of two regions (Tatarstan and Buryatia) in 2022 indicate that, if anything, support for compulsory education has only grown in the interim in those two regions. (NB: bc of design differences, the results aren't directly comparable to those from 1993)
Table illustrating proportion of regional population who supports mandatory titular language education in schools (2022) by ethnicity.
November 19, 2024 at 10:27 PM
kailmarkvart.bsky.social
A key element of this story is that regional language policies have been - and remain - popular. In eight of the 15 regions surveyed in the 1993 Colton/Hough survey, a majority of respondents supported compulsory education in regional (titular) language.
Table illustrating proportion of regional population who supports mandatory titular language education in schools (1993) by ethnicity
November 19, 2024 at 10:27 PM