Talking to voters about LGBT issues changes bias

Ran one of the many headlines this week. “Hang on”, I though, “this seems awfully familiar…”

If it feels familiar, it’s because we have heard this story before. Specifically, in late 2014 when Michael LaCour and Donald Green published a flashy article in Science claiming a short conversation with a door-to-door gay canvasser was enough to change people’s opinion towards gay marriage, and maintain that opinion nine months later. The study received wide coverage, both for the large implications for political campaigning and reducing discriminatory bias, but also for the feel-good factor of the power of human interaction. “At least”, exclaimed the campaigners, “here is science proving human connection can break down prejudice”.

Alas, it was not meant to be. In May the following year, Broockman and Kalla, two graduate students at University of California, Berkeley published a comprehensive critique of LaCour’s paper. A thorough statistical analysis of the original data, as well as following up on the survey companies used for the study revealed several irregularities, which were hard to explain without accepting the data had been falsified. Shortly afterwards, the original paper was retracted.

So far the story is unusual, but not surprising. Cheating in science happens on occasion, particularly in the glamorous, high impact journals like Science. The story was a disappointment for LGBT campaigners, but a small victory for open science and the public availability of data. And that should have been the end of the story, except for those two graduate students back in Berkeley. Broockman & Kalla’s were deeply interested in the LaCour paper in the first place (they did write a 27 page analysis on it, after all), not because they set out to debunk it, but rather because they were planning their own follow-up study on canvassing techniques and opinion shifting.

That follow-up study is now published, and much to everyone’s surprise it seems to broadly agree with the trend in the (allegedly fraudulent) original paper. This new study shows that a 10-minute conversation with a canvasser can change people’s opinion about transgender issues in a similarly long-lasting way. But importantly, and here is where the study differs from LaCour’s interpretation, this effect is seen even if the canvasser are not transgender themselves, changing the focus of why the effect occurs. The original study was all about humanising an issue – by exposing voters to real, flesh-and-bone gay activists their opinion can be shifted by linking the issue to the person at their door, or so the logic ran. With their latest findings, Broockman suggests instead that the success of canvassing relies on a particular perspective-taking technique, and not with exposing prejudiced individuals to a gay or transgender canvasser to humanise them.

Many of the positive design features in the new study came about from the LaCour debacle. Recruitment techniques, canvassing methods and the perspective-taking approach were all informed by the ensuing debate, fine-tuned over the course of the fallout, and it could be argued that this new study wouldn’t have been as successful without it.

This story is therefore a tale of good science. By making the data for LaCour’s original study public, the irregularities were exposed and a better study came out of it. We found out the original interpretation was likely inaccurate, and now have a better idea of what works when changing people’s opinions (whether the interpretation of the original data has any validity in light of the alleged fraud remains an open question).

Most tellingly, Broockman and his colleagues remain committed to open science, having seen the benefit of inspecting other scientist’s data. They have published all of the data and code for their study, in case any amateur sleuth wishes to take a magnifying glass and scrutinise their findings – and, as we have learned, that can be a very good thing indeed.

—————————————————————————————————

Broockman & Kalla (2016) was published in Sciencealong with accompanying commentary. Some great coverage of the story has featured in NPR, FiveThirtyEight, and Wired.

Advertisements