Ben Tappin

Ben Tappin

09-11-2022

22:01

New WP! There’s been lots of concern about political microtargeting (e.g. Cambridge Analytica). But does it work? We aimed to quantify the persuasive advantage of p. microtargeting over other messaging strategies in a US issue advocacy context. 1/XX

“Political microtargeting” refers to the practice of sending different types of political messages to different types of people in order to maximize persuasive impact, and has been facilitated by the rise of digital and social media (though it was also in use beforehand).

Despite these concerns, the extent to which political microtargeting confers a persuasive advantage over alternative (non-targeted) messaging strategies is pretty unclear on the basis of existing evidence. Why? There are several reasons.

For example, most previous studies of political microtargeting do not directly compare its impact to that of alternative messaging strategies being used by campaigns. Yet this comparison is clearly important for understanding the persuasive advantage conferred by microtargeting.

Another reason is that most existing estimates of the persuasive impact of political microtargeting come from studies with small samples of messages. This makes it difficult to reliably detect whether or not different people are best persuaded by different types of messages.

In our paper we aimed to address these issues by (i) examining a large sample (i.e., dozens) of different persuasive messages, and (ii) using randomized experiments to directly compare the persuasive impact of microtargeting against two alternative campaign messaging strategies.

In study 1, we consider a campaign whose aim is to sway public opinion on one of two policy issues: (i) support for the U.S. Citizenship Act of 2021—a legislative bill proposed by President Joe Biden on his first day in office—or (ii) support for a Universal Basic Income.

In a Calibration Phase, we train machine learning models on existing experimental data to learn which messages are most persuasive for which types of people. We also use this data to identify which message is the most persuasive overall (for the single-best-message strategy).

Then, in the Experiment Phase, we conduct a new experiment in which people are randomized to one of the three messaging strategies (or a control group), determining whether they see a microtargeted message, the single-best-message, or a message at random (naive strategy).

The study 1 results suggest a sizable persuasive advantage to political microtargeting. The precision-weighted ATE under microtargeting was approx. 70% larger than single-best-message (5.96 vs. 3.48, p = 0.004) and over 200% larger than naive (5.96 vs. 1.79, p < 0.001).

In study 2, we explored the generalizability of this result to a less typical political advocacy context—in which an issue advocacy campaign chooses not only which message to show people on a given issue, but also which issue attitude to target (of many different issues).

In study 2, microtargeting could provide a large persuasive advantage if, for example, different types of people are more or less persuadable on different types of issues. We used the same Calibration/Experiment Phase procedure as in study 1, but on different data.

The study 2 results somewhat departed from those of study 1: The ATE under microtargeting was slightly smaller than the single-best-message strategy (12.30 vs. 15.42, p = 0.11), but still approx. double the size of the naive messaging strategy (12.30 vs. 6.19, p = 0.008).

In sum, we found that, in a typical issue advocacy context—in which issue campaigns are advocating for or against a particular policy issue (study 1)—microtargeting exceeded the persuasive impact of alternative messaging strategies by an average of 70% or more.

In a less typical context, where issue campaigns have flexibility not only on the message but also the issue (study 2), microtargeting conferred a weaker advantage. This could be due to people being most persuadable on the same types of issues (but there are other explanations).

Generalizing these results to real-world political campaigning is difficult (see excerpt). Moreover, we would like to see our method and results extended to other policy issues, as well as candidate persuasion, to more fully understand the potential advantage of microtargeting.

On that point, we are currently searching for datasets to include in a microtargeting meta-study. If you know of an experiment where participants were randomized to see one of at least five different persuasive messages, we would be delighted to hear about it!

Final tweet. If you’re still reading at this point—much obliged! My coauthors (Chloe Wittenberg, @lukebeehewitt @AdamBerinsky @DG_Rand) and I would be grateful to hear your feedback on this work. (Thanks to @RobbWiller and Jamie Druckman for already providing helpful feedback)



Follow us on Twitter

to be informed of the latest developments and updates!


You can easily use to @tivitikothread bot for create more readable thread!
Donate 💲

You can keep this app free of charge by supporting 😊

for server charges...