University of Zurich Researchers Conducted an AI Persuasion Experiment on Members of This Online Community, Without Consent

In March, the volunteer moderators of the Change My View subreddit learned that researchers at the University of Zurich had been covertly conducting an experiment on their community members. By injecting AI-generated comments and posts into conversations, the researchers had wanted to measure the persuasiveness of AI.

There was one big problem: They didn’t tell community members that they were being experimented on. They didn’t tell the community moderators. They didn’t tell Reddit’s corporate team. Only when they were getting ready to publish, did they disclose their actions.

It then became clear that beyond the lack of consent, they had engaged in other questionable behavior: Their AI-written contributions had spanned multiple accounts, pretending to be a rape victim, a trauma counselor focusing on abuse, a Black man opposed to Black Lives Matter, and more.

Community response was swift: Overwhelmingly, members were unhappy. The moderators insisted the research not be published. Reddit threatened legal action. Initially, the researchers were defiant but eventually, they apologized and pledged not to publish the research.

Change My View volunteer moderator Logan MacGregor joins the show to discuss what went on behind the scenes, plus:

• The danger of publishing the research
• Reaction to the apology
• How AI is going to challenge the idea of trusting an online community

Big Quotes

Blame the manipulators, not the members and moderators (1:49): “Manipulation in online communities has existed forever. What’s happening with [AI is] the believability, the speed at which people can do it. … The fault always rests with the person who chooses to manipulate the community. It’s easy to fool people … and to do something that undermines the trust of something. It’s harder to build trust.” -Patrick O’Keefe

Why a promise not to publish was important (13:21): “From my perspective, I think the things that we wanted the most [from the researchers were] an apology and a promise not to publish. The second was really important because we were concerned that if this was published in a peer-review journal … if it was elevated to a prominent journal, that our community, which is supposed to be a protected human space, would now become just another sandbox for researchers. We felt very strongly that it should not be published. … Unfortunately, it didn’t land well.” -Logan MacGregor

When a community leader stands for their community, they often stand for all communities (14:52): “When one community person – a volunteer, a host, a person in this line of work – stands up for their community, they stand up for all communities.” -Patrick O’Keefe

Just because bad comments exist online doesn’t mean new ones won’t cause harm (20:10): “So much of what [the researchers] did to try to prevent harm was to say ‘comments like this happen all the time online, we don’t think that it’s going to cause individual trauma.’ We kind of dispute that because some of the comments are [you] pretending to be a trauma counselor and maybe that could actually cause some harm. … I don’t think they thought enough about community impact until after the community screamed ‘ouch.'” -Logan MacGregor

You can’t just blame AI for this (22:52): “One thing that’s really special about Change My View is that it’s a human space; it’s a decidedly human space. … The University of Zurich is a decidedly human space. What I think is so insidious about AI is it’s caused people to behave in ways that I don’t know we would have, without the stupid thinking machines. Because it’s a toxic influence. Unlike the bots that are invading us daily, that we’re constantly shutting down. …

“That hurts a little bit more than just dealing with bots, because this wasn’t just bots. These are people interacting with other people, and there was a human element there. The researchers