The disturbing trend of state media use of deepfakes

Social media has been awash with fake images of a stylish Pope Francis, Elon Musk protesting in New York and Donald Trump resisting arrest.(Pixabay photo)

Social media has been awash with fake images of a stylish Pope Francis, Elon Musk protesting in New York and Donald Trump resisting arrest.

Such AI-generated images and videos, or deepfakes, have become increasingly accessible due to advances in artificial intelligence. As more sophisticated fabricated images spread, it will become increasingly difficult for users to differentiate the real from the fake.

Deepfakes get their name from the technology used to create them: deep-learning neural networks. When unleashed on a dataset, these algorithms learn patterns and can replicate them in novel — and convincing — ways.

While this technology can be used for entertainment, it also has dark potential, raising social and ethical concerns.

Unlike simple stories or memes which differ little from propaganda techniques used by Nazi Germany and photo editing by Communist Russia, deepfakes have a high degree of realism. Their accessibility to the public and states could erode our sense of reality.

Fake news anchor

Beyond the growing concern that AI-generated art threatens human art and artists, deepfakes can be used as the unchecked mouthpieces for organizations and states.

Leading the way, China’s state media has experimented with an AI news anchors, named Ren Xiaorong. Ren, although not the first AI news anchor developed by in China, illustrates both the commitment to the technology and the incremental increases in realism.

Other countries such as Kuwait and Russia have also launched AI generated anchors.

When looking at these anchors, we might object that only the most naive viewer would mistake them for real humans, such as Russia’s first robotic news anchor. Yet, these technologies are still in their infancy. We cannot dismiss them.

Fabricated news

China’s transparency in using AI-generated news anchors stands in contrast to Venezuela’s fabricated news coverage. Venezuelan state media presented favourable reports of the country’s progress, purportedly created by international English-language news outlets. However, the stories and anchors are fabricated.

The use of these videos in Venezuela is particularly troubling because they are used as external validation for the government’s activities. By claiming the video comes from outside of one’s country, it provides another source to bolster their claims.

Venezuela is not the only country to adopt these methods. Fabricated videos of Ukraine President Volodymyr Zelenskyy discussing surrender to Russia were also circulated during the ongoing Russia-Ukraine conflict.

Fabricated images and videos are merely the tip of the deepfake iceberg. In 2021, Russia was accused of using deepfake image filters to simulate opponents during interviews with international politicians. The ability to mimic political figures and interact with others in real time is a truly disturbing development.

As these technologies become increasingly accessible to everyone, from harmless meme-makers and would-be social engineers, the boundaries of the real and imagined become progressively indistinguishable.

The proliferation of deepfakes foreshadow a post-truth world, defined by a fractured geopolitical landscape, opinion echo chambers and mutual distrust that can be exploited by governmental and non-governmental organizations.

Disinformation and believable fakes

The spread of disinformation requires that we understand how ideas, innovation or behaviour spread within a social network, referred to as social contagion.

Cognitive science is concerned with “information” — anything that reduces our uncertainty about the actual state of the world. Disinformation has the appearance of information, except uncertainty is reduced at the expense of accuracy.

Observations that disinformation spreads faster that facts likely stems from the fact that when a message is simple, it increases our confidence.

Disinformation spreads for a variety of reasons. It must appear close enough to the “truth” that it is believable. If a new “fact” is incompatible with what we know, we are inclined to reject it even if it is true. People don’t like the feeling of inconsistency and seek to resolve it. People will also ignore the structure and quality of an argument, and focus on the believability of its conclusion.

Deepfakes move us beyond text-based persuasion, because images makes a message far more memorable — and persuasive — than abstract concepts alone. Its use in spreading disinformation is therefore far more concerning.

The structure of the environment is also critical. People attend to available information, focusing on information that confirms their prior beliefs. By increasing the frequency of images, ideas and other media, we increase people’s confidence in their own knowledge and the illusion of consensus.

Social networks and contagion

While we look for credible source of information — experts or peers — our memory stores information separately from its source. Over time, this failure of source monitoring results in our retrieval of information from memory with understanding its origin.

Through product placement and algorithms that control our exposure to media, marketers and governments have exploited these techniques for generations. Most recently, social media influencers have been paid to spread disinformation.

The introduction of AI will only accelerate this process by permitting tighter control of the information environment through dark patterns of design.

Legal, social and moral issues

Producing, managing and disseminating information grants people authority and power. When information ecosystems become flooded with disinformation, truth is become debased.

The accusation of “fake news” has become a tactic used to discredit any argument. Deepfakes are variations on this theme. Social media users have already falsely claimed that real videos of U.S. President Joe Biden and former U.S. president Donald Trump are fake.

Social movements such as Black Lives Matter or claims about the treatment of the Uyghurs in China rely on the compelling qualities of videos.

Once we form a belief, it is difficult to counter. The time required for verification — especially if left to the user — allows disinformation to propagate. Private and public fact-checking websites can help. But they need legitimacy to foster trust.

Brazil provides a recent demonstration of such an attempt. After the government launched a verification website, critics accused it of pro-government bias. However, government officials noted that the site was not meant to replace private initiatives.

There is no simple solution to unmasking deepfakes. Rather than passive consumers of media, we must actively challenge our own beliefs.

The only way to combat harmful forms of artificial intelligence is to cultivate human intelligence.

Jordan Richard Schoenherr, Assistant Professor, Psychology, Concordia University

This article is republished from The Conversation under a Creative Commons license. Read the original article.