Real or fake? How to spot misinformation

Descriptive transcript

Hello curious people. I'm Dr Marian-Andrei Rizoiu. I'm an Associate Professor in the Behavioural Data Science Lab here at UTS. I'm here to answer your curious questions about the spread of misinformation in our digital world. This is Social Curious.

Our UTS community have sent in some thought-provoking questions to tackle. So let's get started.

This is a very good question. In fact, social media is our first attempt—or it's the first time—when we are mediating social interaction using a digital technology. And digital technologies, as you know, they run on algorithms. There's plenty of them: how we interact, how we save our data, the type of data that we work with. But one of them is by far the most important, shaping a lot of the interactions online, and that is the online recommender engine.

The purpose of the recommender engine is to help us navigate the digital landscape of our online social media by recommending content that we might prefer. For example, you may be on your favourite video content streaming platform and you don't know what to watch tonight. So what does the recommender engine do? Well, it analyses your previous views. It also looks at how similar you are to other viewers on the platform. Maybe you have friends on the platform, so it also looks at what they watch and what they prefer to watch. It aggregates all of that and recommends to you a couple of pieces of content for you to watch. And that is how it works.

Where do we also see recommender engines? Well, you do your shopping online. Maybe you're buying a new gadget. When you're searching for that gadget, there are always going to be recommendations of "see also" or "frequently bought together". The same principles apply there. The recommender engine looks into what you intend to buy and then aggregates preferences of previous buyers together with your own personal taste, and creates a personalised recommendation.

Is there a formula for viral content? Now that is a great question. And before we can actually tackle that, before I can tell you about viral content, I actually have to tell you why interactions on online social media are special and why they are mimicking some of the things we've already seen in the offline world.

So the way information travelled in the offline world, back in the day, was that people would tell news to their friends and families, and they would go on and tell it to their own friends. In digital media and on the platforms, we are actually mimicking the word-of-mouth process—a digital word of mouth.

And what better way to explain how the word of mouth actually works digitally than by using these lovely offline real-world icons. So let's assume that you're writing a Facebook post about a new hobby. The moment you post, people around you—your friends—will then see it in their feeds. They will like it, they will hit the like button or they will comment. What that leads to is that by liking it, they push that content into the feed of their friends who are not directly connected to you. So that leads to a second-generation spread. And their friends will spread the content to their own friends, therefore leading to increasing number of generations—therefore, the digital word-of-mouth process.

Now, the formula for viral content—well, we don't really know exactly how it works because it's a mix of two things: it is a mix of quality, and it is a mix of pure luck and timing. So yes, if the original content is of a high quality, of course it will spread more because people are more inclined to like it and therefore push it in their networks. But also, who your friends are is very important. If one of your friends is a highly followed person—say, an influencer—they will have thousands, maybe tens of thousands of followers. Therefore, the likelihood of the content spreading more widely and wildly is going to increase.

So what is the key to virality? Well, writing good content is good, but having influential friends is even better.

Why does social media feel so extreme? Well, that is actually through the design of social media. Now, remember that it is a digitised approach to information exchange. And back in the day—go back a couple of decades, maybe 20 or 30 years—the people we would interact with most were the people physically close to us. So if you lived in a small town, that would be your family, your friends, maybe the people you bumped into at the grocery store.

But these days, with online social media, we can discuss with people on other continents in real time. What that means is that we get exposed to all sorts of views in real time.

And we also know that on social media, most people don't really post. We call these lurkers. They tend to consume content, but they are not very vocal. However, there is a very vocal minority out there. But now everyone is suddenly tuned to these vocal minorities, which may make them seem larger than they are.

Add on top of it the fact that there are what we call homophilic links, meaning that we tend to structure in groups based on preferences. We will surround ourselves with people that are similar to us in taste and worldviews, and we create Facebook groups and WhatsApp chat groups where we tend to exchange views. And in these safe environments, sometimes that leads to the proliferation of all sorts of online toxic content.

How do biases drive the spread of misinformation? Misinformation is defined as information which is not accurate or fact-based. But typically, it is spread not on purpose. So this would be people spreading information because they truly believe it's true. In fact, it doesn't have any scientific backing.

Now, add on top of it the fact that we know that more than 60 percent of the adult population actually consumes news from online social media. They no longer go to the websites of the major news publishers, but they will consume the news from social media. But where do they spend most of their time? They spend most of their time in these homophilic bubbles. So they essentially consume the information that is posted by their friends.

These bubbles not only serve you information with this particular slant, but they also act essentially as filters—we also call them filter bubbles. What that means is that the alternative views of the information never make it within the bubble. So they essentially shield the members within the groups from other types of information, which means that in time, these groups can slide into more extreme views and sort of disconnect from the main narrative.

Is it possible to create an ethical algorithm for social media platforms? The answer to that question is sort of yes. We could, of course, ask the recommender engine to abide by certain principles. However, we need to also understand that any constraints that we add to a recommender engine actually reduce its overall efficacy.

The reason for this is that the purpose of the recommender engine is to optimise one single measure. So it is trained, usually using machine learning approaches, to perform optimally because it is a single measure. But now, if we want to introduce additional ethical directives, it needs to essentially optimise multiple measures at the same time, which means that the solution it arrives at will not be the best solution on every single dimension in isolation.

What does that mean? So, on its original task to increase engagement, therefore increase monetisation, an ethical approach is likely—not impossible—to perform with lower efficacy. That means that there is very little monetary incentive for the platforms to actually do it, because it reduces their financial position.

How do social media algorithms make false information spread more? We need to start by acknowledging that the recommendation algorithms, and algorithms in general, do not prefer to spread misinformation. So they are not designed to spread misinformation. In fact, they are not designed to spread any particular type of information. They are designed for one thing only, which is increasing engagement. But it will feed us the information we engage most with.

So if we tend to consume and engage more with a particular slant of the information—say, with misinformation—it will get noticed by the recommender engine, which in turn will start recommending more of the same because, well, it is more engaging. And that is one of the pathways through which automatic recommender engines can actually end up reinforcing the spread of misinformation.

What steps do platforms or creators take to counteract misinformation claims? Well, there's a number of measures that platforms can take, and they vary from, say, hard moderation, de-platforming, or soft moderation.

Hard moderation essentially means removing content from the platform—that's typically restricted to when content is clearly toxic or illegal. Then, de-platforming usually means users are suspended from the platform. Again, it's reserved for people who break the terms of service or perform illegal things. So these are pretty harsh measures, reserved for when it is quite obvious that the activity is wrong. And misinformation is actually a continuum, so it is difficult to apply these strong measures on things that can be classified as humour or sarcasm.

Another measure which tends to work a bit better in this case is what we call soft moderation. Soft moderation essentially puts a warning on top of the content that contains claims that have been debunked, together with a link to the debunking.

Now, all of these are what we call reactive measures. So the misinformation arrives and then we do something to respond to it. However, what is known to work best is pre-emptive measures. Pre-emptive measures essentially mean teaching users to use critical thinking to stem misinformation before it reaches them.

You're a data scientist. What worries you most about misinformation and deepfakes?

My main source of worry stems from the reach of information. So remember when we discussed that using online environments, you can get real-time access to almost any type of information out there, including the more extreme views. Now, repeated exposure to such more extreme types of information will lead to changes in our perception, which then at the societal level actually leads to what we call offline effects.

Things like we lose our trust, or perceived trust, in our democratic institutions because increasingly extreme and vocal voices online keep bombarding us with that information. It's not what the majority necessarily thinks, but we keep getting exposed on and on to that type of information.

Now, deepfakes and generative technologies in general open up a lot of opportunities for us, but they also open up challenges. Because nowadays, technology is accessible to almost anyone with minimum to medium technological skills, which now means that almost anyone can produce a very believable deepfake of a politician or a public figure saying outrageous things. We've already seen this repeatedly with political figures. So that will only contribute and make the mistrust even worse, because now we are no longer sure about the information, but not even the content that we're seeing—we can't be sure that that political person actually said the things that the video says.

And the final point is actually being exposed to proper extremist content, which would normally—even two or three decades ago—it would have been quite rare to be exposed to such content. But these days, it's out there and it is most worrying for the young generation—for the children, the teenagers—who barely open access to the internet and they are already bombarded with toxic masculinity, extremist views, dangerous teen challenges. These are all very toxic content that exists out there freely, and they can access it with just a computer in their room. That's what worries me as a data scientist.

That was all the questions for today. I hope you learned something new. Until next time, stay curious.