Can You Believe Your Eyes? Deepfakes and the Rise of AI-Generated Media
As artificial intelligence (AI) continues to impact nearly every industry, much attention is given to its ability to recognize patterns in information—e.g., face recognition, document review, and fraud detection—as well as make decisions. Recently, however, another capability of AI has attracted attention: its ability to create new content. This ability exists in a type of AI known broadly as generative models, which learn from data samples provided to the model and can use that knowledge to create, among other things, new, hyper-realistic media content.1
One of the most visible and concerning applications of this capability is so-called “deepfakes,” online videos in which a participant’s face is swapped with that of another person.2 Although fake videos have been within the realm of possibility of deep-pocketed studios for some time, the current deepfake phenomenon began when an anonymous developer released a software tool based on existing, open-source components to create convincing face swaps with only moderate effort using consumer-grade hardware.3 Almost immediately, some online communities began using the tool to create pornographic videos substituted with the faces of various celebrities.4 Major online platforms acted quickly to prohibit the videos and shut down some of the communities, citing existing prohibitions against nonconsensual pornography. However, the technology is still widely available, and those users have largely moved to alternative platforms.5 The numerous issues posed by these nonconsensual videos are likely to continue as the technology improves, reducing the required number of training samples, amount of computing resources, and expertise needed to create a convincing fake.
The challenge posed by deepfakes goes beyond the current craze of nonconsensual pornographic videos to include potential threats to national security and democratic institutions.6 Using the technology, it may be possible to create hyper-realistic fake videos of government officials, political candidates, or foreign actors engaged in illegal or unsavory activities in an effort to harm them or to provoke a response.7 For example, a prominent politician could be convincingly substituted into footage of an illicit meeting or shown taking a bribe.8 In line with the current national conversation on “fake news,” these types of fake videos could spread widely through social media and would be immensely difficult to effectively repudiate once distributed.9 Incendiary fake videos could have the potential to sway elections, create popular outrage, or even provoke violence in the form of civil unrest or unjustified military responses.10 Beyond the immediate, tangible harms in response to a particular fake video, the proliferation of fakes—and the ever-growing ease of creating them—will have lasting effects as the public loses trust in media content from the constant tug-of-war between sophisticated fakes and attempted rebuttals.11
In addition to the obvious issues presented by video face-swapping and sophisticated, AI-created audio forgeries,12 the computational creativity enabled by generative models has other, less-obvious impacts. For example, researchers have created models that can transform features of existing footage in unexpected ways, such as converting a winter scene into summer or a daytime drive into nighttime, complete with computationally generated streetlights.13 Taking this creativity one step further, researchers have trained models to create entirely original content based on the training samples.
Often, researchers use an approach known as generative adversarial networks.14 This approach pits two artificial neural networks against one another—one trained to generate content and another trained to spot authentic content.15 As both networks continue to learn and improve, the “generative” model begins to produce progressively higher quality fakes.16 This technique has been used to create new scenery, including photos of bedrooms based purely on characteristics of other rooms encountered by the model.17 One of the most intriguing examples comes from the hardware manufacturer Nvidia Corp., which created “fake celebrities”: photorealistic images created to mimic the qualities of real celebrities without actually being based on a particular person.18
Lawmakers, already weary of the explosion of fake news, have shown some concern in the face of this new technology and its impact beyond politics.19 At least one congressman has suggested that the federal government can take a larger role in preventing the creation and proliferation of fake content, for example, by tasking the nation’s research apparatus to create new technologies to authenticate content or by regulating authentication on internet platforms.20 Others have introduced bills designed to improve transparency and traceability on internet platforms, with the hope of limiting the unchecked proliferation of fake content.21 Depending on the technological approach used to address these problems, there may be significant implications for online privacy.22