Make Nations Great Again? The Power of AI and Populism

PPA Talk Series: Measuring Populism and its Sub-Dimensions Using Large Language Models

Professor Kenneth Benoit, Dean of the School of Social Sciences and Professor of Computational Social Science at Singapore Management University, will deliver a presentation titled “Measuring Populism and Its Sub-Dimensions Using Large Language Models” on the afternoon of October 3rd at the HKU campus.

Populism has emerged as a pervasive phenomenon, manifesting primarily among radical right and radical left political actors and citizens, and influencing democracy, political culture, policy-making, and political behaviour.

A key challenge in studying populism—and its core conceptual components, such as people-centredness, anti-elitism, and a Manichean worldview—lies in effectively analysing political actors’ texts, including manifestos, speeches, and social media posts. Although human coding has been a reliable method, it is too resource-intensive to scale to large corpora and requires both domain-specific knowledge as well as the ability to read the texts in their original languages.

Automated text-as-data approaches, on the other hand, frequently fail to detect populism and its sub-dimensions, often yielding low accuracy even in prototypical cases. We propose a solution using GPT-class Large Language Models (LLMs) using instruction tuning to mimic human coding behaviour, but at scale and without natural language limitations or any requirement for pre-processing or even document conversion. Unlike earlier generations of machine learning approaches, LLMs can accurately identify populism and its sub-dimensions, even when facing a needle-in-the-haystack problem of low-frequency populist statements. We validate our approach by comparing LLM-derived populism scores with established expert surveys on party-based populism and populism sub-dimensions.

Chaired by Professor Robert Thomson, head of Department of Politics and Public Administration, this event is open to both the public and the HKU community

About the speaker

Kenneth Benoit is Dean of the School of Social Sciences and Professor of Computational Social Science, Singapore Management University. He has previously held positions at the London School of Economics and Political Science, where he was Director of the Data Science Institute and before that, Head of the Department of Methodology; Professor (Part-time) in the School of Politics and International Relations, Australian National University; Professor of Quantitative Social Sciences in the Department of Political Science at Trinity College Dublin; and at the Central European University (Budapest). He received his Ph.D. (1998) from Harvard University, Department of Government with a specialization in applied statistical methods. His current research uses large language models and advanced machine learning methods for analyzing large amounts of textual data, mainly political texts and social media. His substantive research in political science focuses on comparative party competition, the European Parliament, electoral systems, and the effects of campaign spending.