48 States and counting now make it illegal to alter images or videos illicitly

Giving it to you straight, cross my heart, totally coming clean, but as of September 2025, with Michigan joining, nearly every US state has passed some form of legislation outlawing the production or distribution of nonconsensually digitally manipulated media, i.e. deepfakes.

“These videos can ruin someone’s reputation, career, and personal life. As such, these bills prohibit the creation of deep fakes that depict individuals in sexual situations and create sentencing guidelines for the crime,” Michigan Governor Gretchen Whitmer’s (D) press release stated. 

Sponsored by State Representatives Bierlein (R) and Tsernoglou (D), House Bills 4047 and 4048 mandate creators as well as disseminators of deepfakes that “would cause physical, emotional, reputational, or economic harm to an individual falsely depicted” be imprisoned for no more than a year and/or a $3,000 USD fine.

But the potential repercussions don’t stop there. For instance, if the person who created the deepfake did so to for monetary gain, uploaded it to their own or any other site, was previously convicted of any other crime, or  if the deepfake was made to “harass, extort, threaten, or cause physical, emotional, reputational, or economic harm to the depicted individual” what was previously a misdemeanor automatically becomes a substantial felony.  

Book ‘em, Danno

Though the Michigan bills were drafted to make individual deepfake instigators responsible for their actions and don’t target the online platforms involved, the federal Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Network (TAKE IT DOWN) act does.

In May this year, Ted Cruz (R) introduced this legislation that sets what many critics claim is an unrealistically tight deadline for platforms to identify and remove nonconsensually digitally altered content. 

RECOMMENDED READ: Uncool at Any Speed: Deepfakes Are Emotionally Harmful and Environmentally Wasteful

TAKE IT DOWN has also been lambasted as too-far-reaching.  Alarming enough, but potentially more so in light of the current US administration’s propensity for retribution towards any perceived opponents, which could easily turn it into an anti-free-speech weapon.

As the Electronic Frontier Foundation wrote about it back before it was signed into law:

The takedown provision applies to a much broader category of content—potentially any images involving intimate or sexual content—than the narrower NCII definitions found elsewhere in the bill. The takedown provision also lacks critical safeguards against frivolous or bad-faith takedown requests. Lawful content—including satire, journalism, and political speech—could be wrongly censored.

These are their stories

Sex-related deepfakes cause devastating social, financial, and emotional harm, yet state and federal laws to ban them haven’t considered how difficult it can be to tell the difference between what is real and what is not.

In addition to technical challenges platforms must also be willing to put themselves on the line when it comes to spotting illicit content.

As Kate Ruane, the Center for Democracy and Technology’s Free Expression Project director, told 404Media,  “If you do not have perfect technology to identify whatever it is we’re calling a deepfake, you are going to get a lot of guessing being done by the social media companies, and you’re going to get disproportionate amounts of censorship.”

Further adding, social media companies may instead prefer just to ban everything that might be a deepfake instead of running afoul of laws like TAKE IT DOWN: “For a social media company, it is not rational for them to open themselves up to that risk, right? It’s simply not. And so my concern is that any video with any amount of editing, which is like every single TikTok video, is then banned for distribution on those social media sites.”

To protect the innocent

As we’ve written before, though it’s critically important to do whatever’s necessary to prevent the creation and propagation of sexually-assaultive deepfakes, we need to ensure these laws do what they are intended to do and don’t end up making an already dire situation far worse.

A great place to begin is to ensure lawmakers understand how technologies like deepfakes actually work and, from there, take the expert advice on what can, can’t, and should never, ever be done to stop the spread of deepfakes, while at the same time protecting the civil liberties of others.

Image Sources: Depositphotos

The post The Real Deal: Laws Against Sexual Deepfakes Spread Across the US appeared first on Future of Sex.

From Taylor Swift nudes to draining resources—AI’s dark side is on full display

John Ladarola of The Damage Report may have said it best, “Elon Musk’s favorite baby is up to no good again” and by baby he meant Grok, now known as SuperGrok as a paid service.

Whether making false claims of “white genocide” in South Africa or labelling Musk a “top misinformation spreader”—challenging his maker to shut him down—to spewing 16 straight hours of antisemitism while branding itself as MechaHitler, Musk’s unfettered AI ubermensch has garnered a lot of negative publicity! 

But it’s as a multi-tasker that Grok shows its versatility. This weekend alone, a new version of Imagine, Grok’s generative AI video tool, spat out 34 million images and videos in 48 hours. And some of them consisted of nonconsensual deepfakes of celebrity women in scanty clothing or no clothing at all, including a nude deepfake video from the prompt “Taylor Swift celebrating Coachella with the boys,” without the user even asking for nudity.

“Custom,” “Normal,” “Fun,” and “Spicy”

Jessica Weatherbed tested the new app by entering the Swift-at-Cochaella prompt, choosing Spicy over the other three settings. As she wrote for The Verge:

While image generators often shy away from producing recognizable celebrities, I asked it to generate “Taylor Swift celebrating Coachella with the boys” and was met with a sprawling feed of more than 30 images to pick from, several of which already depicted Swift in revealing clothes.

Grok, of course, is also the AI Musk installed in the US federal government in January, without congressional approval or oversight. As far as anyone knows, Grok may still be having its way with the personal data of taxpayers, veterans, social security recipients, Medicare and Medicaid recipients. and the like, generally wreaking havoc and destroying necessary government services and programs. 

Essentially we have the most powerful digital adolescent in the universe sulking in a dimly lit server, smashing real world lives and real world premises that don’t belong to it, while also generating spicy content for its users–perhaps even government higher ups, who could then be susceptible to AI blackmail—all the while gaslighting the hell out of us in the name of free speech and government efficiency.

Can’t tell the difference?

Grok’s Imagine may have the above presets, but it can’t seem to tell the difference between them. After all, even an LLM can struggle with multiple meanings and nuances, right? 

What’s a bit of spicy fun for one user might be not-so-fun for others, like the people on the other end of the deepfakes, celebrity or not. The point is, AI must have guardrails and safeguards to prevent the enormous damage that results when developers don’t infuse their creations with alignment to the public good.

Weatherbed’s experience of getting more than she asked for or wanted is not uncommon. Unwanted exposure to sexualized content has been a problem for Grok users all along. For example, one reviewer of an older version noted that a “creepy live sex doll” had been added to their roster of AI companions, without consent, along with a “teddy bear for kids.” An unnamed developer responded “hey, you can always turn companions off in settings.” Another reviewer wondered “when did NSFW become default ‘on’d?” 

The 1.1.43 Grok Imagine app, launched in mid-August, is rated for ages 12+ but requires no documentation of age 18+ status beyond clicking a box. According to the Grok website, “Some subscription levels promise unlimited AI-generated images, perfect for creatives, designers, and content teams.” 

My body, my image, my choice

How can an AI even create a celebrity deepfake? The images used must certainly be based on photographs and illustrations created by human beings and fed into AI training. Like most creators, they have probably not been compensated for use of their work, just as Taylor Swift and other notable people are not compensated for the facsimile use of their face and body. Swift won’t be the first person to file a lawsuit about this, nor will she be the last.

RECOMMENDED READING: How Warm Is Too Warm? ChatGPT4o’s Flirty Female Voice Lands OpenAI in Hot Water

Viewer reactions to The Damage Report included this reminder: “Grok is a word that was invented by Robert Heinlein in his book ‘Stranger in a Strange Land’ so Musk co-opted this word. Is he paying the author’s estate for the use or is it just a rip-off? Either way he didn’t invent this word.”

Another viewer said, “Elon Musk is in the masturbation business. Gross.” 

There is honor among the pro-sextech community—supporting an industry proudly engaged in the masturbation business—which has stated commitments to consensuality among adults. Musk’s rogue AI, the creation of a puerile mentality and corporate culture that seemingly doesn’t know the meaning of the word consent, will never be welcome in our sex-positive communities, nor should it be allowed into science, education, government, public content creation, or any other part of our human cultures.

Plundering resources

Beyond its emotional and social harm, nonconsensual deepfakes are also draining planetary resources at an alarming rate 

Yasmin Kahn, an author and broadcaster told John Ladarola, commented, “this is dumb as hell,” while pointing to the tremendous waste caused by AI technology:

I don’t love that we are using so much of our resources, for the planet…not the digital world, but the actual physical planet. Because I think people forget that even though stuff is existing on the cloud or whatever, you know, that’s so nebulous that people don’t really understand what it means, but things still have to exist somewhere in reality. And where those things exist is in the use of our planet’s resources to power all these things.

The sad thing is, all those deepfake images, whether of high school girls—created by snickering classmates—or of better known adults who can afford to file lawsuits against rich, powerful tech moguls remain cloud-based, ready to potentially damage countless people’s lives, unless we stand up and demand regulations and accountability from generative AI companies. 

Image Source: A.R. Marsh using Ideogram.ai

The post Uncool at Any Speed: Deepfakes Are Emotionally Harmful and Environmentally Wasteful appeared first on Future of Sex.

At least one is out of business, with another possibly to follow

Signaling a turning point toward curbing the proliferation of non-consensual, AI-generated adult content sites, as reported by NPR early this month, MrDeepFakes users were initially greeted with the message, “A critical service provider has terminated service permanently. Data loss has made it impossible to continue operation.” Shortly thereafter, the site has completely vanished, leaving behind nothing but Page Not Found.

Meanwhile, CivitAI, another controversial sex-related deepfake platform, is experiencing its own troubles.  

After Visa and Mastercard cancelled their services with CivitAI, forcing the site to look for alternative payment options, Justin Maier, CivitAI’s CEO, announced, “Civitai is not shutting down.  We have months of runway. The site, community, and creator payouts continue unchanged. We just need a brief boost from you while we finish new payment rails.”

CivitAI may have anticipated the loss of this financial footing, as a month previously, it had implemented a set of new rules aimed at limiting its users’ ability to create sexually explicit LoRAs (Low-Rank Adaptations) of public figures.

Alasdair Nicoll, CivitAI’s Community Engagement Manager, expressed dismay during a Twitch live-stream—as noted by Unite.AI—saying, “These are not changes that we wanted to make. This boils down to new and impending legislation. There’s deepfake laws out there, there’s AI porn laws…payment processors, and ultimately Visa and MasterCard are spooked; they don’t want to be sued, and they’re ultimately driving these changes.”

A law too far or not far enough?

Nicoll may have been referring to the recently signed federal Take It Down Act, which criminalizes the dissemination or threat to disseminate non-consensual user-created and AI-generated explicit content.

Along with forcing sites to delete such materials within 48 hours after being notified by the offended party, it also demands the same for any copies that might remain.

While there’s long been a call for something, anything, to prevent the spread of non-consensual, image-generating explicit sites—and the Take It Down Act may have been partially responsible for MrDeepFakes’ closure and Mastercard and Visa cutting ties with CivitA—others object to the law’s potential conflict with the First Amendment.

Freedom versus abuse

Quoted by the Associated Press, a spokesperson for the Electronic Frontier Foundation, a respected digital rights organization, said, “While the bill is meant to address a serious problem, good intentions alone are not enough to make good policy, lawmakers should be strengthening and enforcing existing legal protections for victims, rather than inventing new takedown regimes that are ripe for abuse.”

RECOMMENDED READ: Do You Hear What I Hear: Deepfakes in Audio Erotica

Furthermore, the law’s insufficient barriers against trivial or biased complaints are troubling, with the EFF adding, “The law’s tight time frame requires that apps and websites remove speech within 48 hours, rarely enough time to verify whether the speech is actually illegal.”

The Cyber Civil Rights Initiative has concerns as well, telling AP the Take It Down Act is “Unconstitutionally vague, unconstitutionally overbroad, and lacking adequate safeguards against misuse.” 

For example, sites might be forced to delete unclothed, non-sexualized images even when they may have journalistic value or erotic, consensual content that happens to offend someone somewhere.

Crimes and punishments

Well-intended, ill-conceived, or both, the Take It Down Act seems to be sending an important message.

Contractors to these revenge porn and non-consensual image generation sites have also taken note.

In the case of MrDeepFakes’ service provider, it may got the message and made the strategic decision to get out while the getting’s good.

On the other hand, though CivitAI may not currently be facing legal troubles, the site is definitely experiencing financial woes due to Mastercard and Visa’s cancellation of payment services.

The bottom line is profit.  Laws are one thing, but what might prove to be deepfake and revenge porn sites’ undoing is when no one will do business with them.

Image Sources: Depositphotos

The post The Real Deal: Deepfake Sex Sites Going, Going—Gone? appeared first on Future of Sex.

Generative AI Isn’t the Villain—Predators and Tech Bro Apathy Are

In December 2024, two teenage boys were charged with creating and sharing 347 deep fake nude images and videos of 60 girls, 48 of them students at a private school in Pennsylvania. According to Rawstory, 59 of the victims were under the age of 18. The images were posted to a server on Discord. One of the boys was a former student.

In the US, explicit AI-generated images of minors are categorized as CSAM (child sexual abuse material). The teen boys who got cheap thrills from exploiting their classmates probably had little idea of just how much trouble they’d be in.

“The number of victims involved in this case is troubling, and the trauma that they have endured in learning that their privacy has been violated in this manner is unimaginable,” District Attorney Heather Adams told ABC27.com.

Susquehanna Regional Police filed five dozen counts of various sex crime charges, including possession of child pornography and sexual abuse of children, according to the same article. 

ABC27.com previously reported that new PA state law, Act125, would not apply to this case as the crimes occured in November 2023 and May 2023. Act125 was signed into law on October 29th and amended existing laws to include any “artificially generated sexual depiction of an individual.” 

The dark side of generative AI

What are deep fakes, exactly? According to NewsNation deepfakes are “video, photo or audio recordings that appear to be real but have been manipulated with artificial intelligence. A deepfake can depict someone appearing to say or do something that they, in fact, never said or did.” 

While it might be amusing to add a photo of the head of a long-dead person, such as Albert Einstein, to the body of a bunny, it is against the law to slap a classmate’s selfie onto a nude or sexually explicit photo of someone else.

Last September, the nonprofit Center for Democracy and Technology (CDT) found 15% of high school students had heard of a fellow student affected by deepfakes. Alexandra Reeve Givens, chief executive of CDT, told Rawstory, “The rise of generative AI has collided with a long-standing problem in schools: the act of sharing non-consensual intimate imagery.” CDT recommends schools update their sexual harassment policies.

Nonconsensually sharing images, whether AI-generated or not, can result in the recipient experiencing further cyberbullying, threats, harassment, and severe damage to mental health. Such images also have the potential to damage college, employment, and dating prospects.

RECOMMENDED READ: Reality Under Siege: Challenges in Detecting and Combating Deepfakes

Girls don’t feel safe at school

Returning to the Pennsylvania case, WGAL8, a local television station, reported parents of 46 of the students have filed a civil suit against administrators of the Lancaster Country Day School, claiming these mandated reporters did not follow up on a complaint made nearly a year earlier, allowing the situation to worsen.

WGAL8 also noted hundreds of students at the school and some instructors walked out protesting the administration’s mishandling of the case:  “One student said she thought the action sent strong messages to the school: that students are upset with how the adults handled the AI situation and that girls don’t feel safe at school.”

Though female students are most at risk for deep fake exploitation it can affect students of other genders too. 

Will the DOJ pursue those responsible?

In November 2024, the Associated Press interviewed Steven Grocki, Chief of the US Justice Department’s Child Exploitation and Obscenity Section (CEOS), about AI-generated child sexual abuse images: “We’ve got to signal early and often that it is a crime, that it will be investigated and prosecuted when the evidence supports it,” He added,“These laws exist. They will be used. We have the will. We have the resources.”

But that was last year. Given the current US administration will Grocki and the CEOS be able to function at the same level as they did in 2024? Is the DOJ’s mission to combat AI-generated CSAM imperiled even as the problem seems to be escalating nationally? 

Federal will and resources may be in jeopardy but at least twenty states have established laws against AI generated deepfake images of minors, including Utah, Idaho, Georgia, Oklahoma, Tennessee, South Dakota, California, and now, Pennsylvania. 

Blame the developers and industry entitlement

Artificial intelligence is just a tool and still somewhat controllable, though for how much longer is subject to debate among those watching its development and developers with growing alarm.

But AI’s potential for existential threat—while important to consider and prevent—is of less concern to the parents of a K-12 child who has been exploited through deepfake images, whether generated by a school peer or an older predator. 

Parents and their children are as much victims of the present culture of entitlement and greed, as they are of the images themselves. It’s clear the AI horse bolted from the barn long before anyone gave the safety and impact of their products a thought. And so hundreds if not thousands of young lives may have been negatively affected. 

The bullies and predators are also to blame. Sadly, they too might be products of entitlement that says it’s okay to turn the body, reputation, and life of another into fodder for their amusement, without consent and without concern for the consequences.

This is the antithesis of a sex-positive, technologically progressive future created by responsible adults concerned with the safety of all, including those under the age of consent.

Image source: A.R. Marsh with Ideogram.ai

The post Deepfake Sex Crimes Surge: When Will Lawmakers and Schools Fight Back? appeared first on Future of Sex.

Reprinted from The Conversation with the author’s permission

After deepfakes of Taylor Swift went viral on the social media platform X (formerly Twitter), Swifties sprung into action. They organized to report violations of X’s “Synthetic and Manipulated Media” policy and flooded the platform with real images of Swift in an attempt to alter X’s algorithm.

The incident renewed calls for federal legislation regarding deepfake erotica. But whether we need to “defeat” deepfakes by censoring and criminalizing it is up for debate—or at least it should be. 

As a criminologist and sexuality studies scholar with expertise in the legal regulation of sex and sexual expression, the push to conflate deepfakes with misogyny and sexual harm is concerning, as is the call for new criminal laws.

Are deepfakes sexual violence?

Deepfake refers to the use of artificially intelligent (AI) machine-learning applications to generate original but “fake” audio, images or videos that may appear authentic. Deepfake pornography (DFP) refers to products that are sexually explicit in nature. 

According to a 2023 report by cybersecurity firm Home Security Heroes, DFP makes up 98 per cent of all deepfake videos online, and 99 per cent of DFP features women. Notably, 94 percent of these women work in the entertainment industry. 

An earlier study of DFP by Deeptrace Labs found that of those in the entertainment industry, most of the 10 most frequently represented individuals were actresses from western countries, followed by South Korean K-pop singers.

Given their gendered, sexual and seemingly “non-consensual” nature, DFP has been widely described as gender-based sexual violence requiring greater civil and criminal regulation of both AI and deepfake porn producers. 

Others, however, suggest that deepfake erotica may be deeply constitutional sexual expression, and that new laws should be put off until more research about the impacts of pornographic deepfakes on those depicted, as well as on internet users, can be conducted. 

Deepfakes raise concerns about false representations—for instance, falsely depicting an individual as sexually active, into a certain type of sex or involved in the adult entertainment industry. Whether this constitutes sexual violence—even if a person is distressed by fake videos of them—is not self-evident.

Many valid reasons exist for why deepfakes may be created and shared, and why it should not be interpreted or criminalized as “image based sexual abuse” or “virtual rape.” These reasons include, but are not limited to the social value of sexual fantasy—including seemingly “deviant” fantasies—and the need to resist prison expansionism and the carceral state.

Need for new laws?

Deepfakes, as with the cheapfakes that preceded them, can be created for malicious purposes including harassment, spreading disinformation and extortion. In instances where the use of one’s image is deeply upsetting to the individual depicted, legal avenues such as civil privacy, intentional infliction of emotional distress, invasion of privacy laws that address “false light” (making false or misleading claims about a person that cause harm to them) and take-down orders may help address their concerns. 

When it comes to deepfake adult content and minors, Canada’s child pornography and its Intimate Images provisions likely apply. And in cases where DFP images and videos are used to harass or extort individuals, laws already exist to address these harms. On or offline, however, there are reasons to resist pro-criminalization strategies.

Queer and sex-radical feminists have long established that even though sex and gender are related, theories of gender oppression cannot wholly explain sex and sexual politics. Importantly, anti-sexual violence feminists and anti-carceral scholars have pointed out the limits and harms of using criminal law to respond to sexual violence for sexual violence victims, the accused and society more broadly. 

Before we can determine whether sexual violence is the best framework for describing and responding to deepfakes, we need a better understanding of deepfake prosumers—those who produce, consume and share their creations—as well as the importance of sexual fantasy, at the individual and collective levels.

Deepfake prosumers

It’s becoming increasingly difficult to interview or conduct research with deepfake prosumers given how widely they are described as depraved, as the embodiment of “toxic geek masculinity,” and as driven by an interest in owning women’s bodies.

Research suggests, however, that many deepfake creators are hobbyists who are more interested in contributing “to the development of such technology as solving an intellectual puzzle… rather than as a way to trick or threaten people.” 

It’s likely that most deepfake prosumers are men, given what we know about the demographics of computer programmers and porn consumers.

Insights from clinical practice suggests that when men do create fake porn misogyny rarely serves as a key motivator. Clinical psychologist David J. Ley observes that more of these cases are “driven by feelings of loss, shame, hope, and fantasy than by misogyny and anger.” Similar to Photoshopped “porno collages,” deepfakes serve as a means to explore fantasies that are likely impossible to fulfill.

But is sexual fantasy a valid reason to create and share deepfake erotica on public and paid platforms? 

Sexual fantasy and deepfakes

Sexual fantasy is more complicated and more important to our sex lives and to our social well-being than people typically realize or acknowledge. For many, sexual fantasy is private and limited to their mind’s eye.

Others, however, see sexual fantasy as something to be manifested as written text, images or digital files, and publicly shared for free or for a fee. 

Critical race feminist scholars have demonstrated that sexual fantasy is both a product of and productive of our complex realities, but that a line can also be drawn between fantasy and reality given the important roles that fantasy plays in our individual and collective lives.

At the individual level, a study that surveyed 1,516 adult cis men and women about their sexual fantasies found that more than half of the respondents—51.7 percent of women and 61.9 percent of men—fantasized about sex with a celebrity.

Sexual fantasy research has also helped establish that few sexual fantasies are statistically unusual or rare. This includes fantasies which have previously been deemed perverted or atypical, such as those that involve violence or humiliation. These fantasies are not only common, but are also “unlikely to be revealing of actual behavior.”

According to Ley, reasons for publicly sharing sexual fantasies range from seeking approval, demonstrating technical prowess and playing with taboo to bonding with those who share the same interests or to arouse others in the way they are aroused so as to feel less alone for having these interests and desires. 

As legal scholar Andrew Gilden writes: 

“The actual process of coming to terms with one’s sexual identity often entails extensive fantasizing, experimentation, education, and social interaction. And these processes are often far less romantic, much less ‘dignified,’ and far less ‘PG’ than envisioned by the evolving legal narratives of sexuality.”

Consent and fantasy

Thinking about deepfake erotica through the lens of sexual fantasy also helps us make sense of the lack of consent in DFP. Consent does not factor into people’s sexual fantasies in the same ways as it does their physical sexual relations: I don’t need permission to fantasize about someone, but I do need permission to have sex with them. 

Consent is a primarily legal term that, at its most general, means voluntary agreement to engage in sexual activity. The use of consent language to refer to the creative process of DFP as image-based sexual abuse or virtual rape shuts down a nuanced conversation about sexual harm and freedom. 

Expanding definitions

Ultimately, sexual fantasy cannot fully explain the phenomenon of deepfake porn. But failing to acknowledge the limits of gender-based sexual violence frameworks comes with its own harms, including the ever-growing definition of sex crime and the expansion of the carceral state.

We need to think carefully about the social and cultural motivations and intent of content creators, in addition to the potential effects of their creations. We need to consider whether expanding the scope of criminal law to address emotional harm in virtual spaces will bring about the changes we want to see, including the reduction of sexual violence. We also need to acknowledge that criminal law has largely failed to prevent, and indeed perpetuates, emotional and physical violence at a level that requires great awareness and care. 

Concerns about sexual autonomy should inform debates about emerging technologies, but alternative frameworks for making sense of and responding to deepfakes should be considered before we censor and criminalize deepfake adult content producers, consumers and products.

Image Sources: Depositphotos

The post Deepfake Erotica Doesn’t Need New Laws, Which May Restrict Sexual Fantasies and Promote the Prison System appeared first on Future of Sex.

Navigating the ethics of AI-generated erotic content

Deepfake technology and nudification services are capable of creating sexualized, nonconsensual images of real people, whether celebrities or unknowns–and a shocking number of these are of women and others presumed female, including minors. 

Celebrity deepfake images, such as those of singer Taylor Swift that flooded the internet in January, have  drawn the most media attention—and views—yet hundreds of thousand, or perhaps millions, of people worldwide have had their careers, personal reputations and brands, social lives, and sense of privacy and security violated.

Sexually explicit, nonconsensual, deepfake images have also become a staple of revenge porn, according to an article published in the University of Cincinnati Law Review.

An Australian ABC News piece , noted most of these sorts of images are created by diffusion models, “a type of generative artificial intelligence model that can produce new and photorealistic images from written prompts.”

Revealing the extent of Non-Consensual Intimate Imagery (NCII)

Though hardly household names, organizations such as Graphika have investigated the deepfake problem and issued reports, such as their early 2023 “Deepfake it Till You Make It.” 

However, their more recent report, reveals the truly massive extent of the undressing images problem:

“A group of 34 synthetic NCII providers identified by Graphika received over 24 million unique visitors to their websites in September, according to data provided by web traffic analysis firm Similarweb. Additionally, the volume of referral link spam for these services has increased by more than 2,000% on platforms including Reddit and X since the beginning of 2023, and a set of 52 Telegram groups used to access NCII services contain at least 1 million users as of September this year.”

Graphika concluded “the primary driver of this growth is the increasing capability and accessibility of open-source artificial intelligence (AI) image diffusion models. These models allow a larger number of providers to easily and cheaply create photorealistic NCII at scale. Without such providers, their customers would need to host, maintain, and run their own custom image diffusion models–a time-consuming and sometimes expensive process.”

Defending reality

According to its FAQ, Reality Defender, founded in 2018, is a “deepfake detection platform for enterprises, governments, and platforms to detect AI-generated content and manipulations across audio, video, image, and text files.” Unfortunately it is only accessible to large organizations, businesses, and governments. 

Though Reality Defender’s detection services are not available to the average consumer, it’s helpful to look through the website’s blog posts. For starters, I recommend “A Brief History of Deepfakes”, paying special attention to this quote:

“No history of deepfakes can be complete without acknowledging contributions made by the average internet user. The open-source deepfake creation tools have been tested and refined by legions of hobbyists, who have utilized these tools for purposes of benign entertainment (memes, swapping out actors’ faces in classic movies) and more sinister, appalling goals, like the creation of deepfake pornography.” 

“It was the participation and driving interest of everyday users, beginning in 2017, that has brought the technology to where it is now. This continued democratization of such powerful tools demonstrates the dire need for adequate countermeasures to detect and isolate deepfakes before they can be used for malicious purposes.”

So while average internet users have the ability to contribute to the development of deepfakes, other average internet users have no way to protect themselves from them.

Calls for regulation, social responsibility, and technical safeguards

In the US, politicians appear to be finally paying attention. In September 2023, Representatives Yvette D. Clarke (D-NY) and Congressman Glenn Ivey (D-MD) introduced the DEEPFAKES Accountability Act of 2023.

According to the Los Angeles Times, Senators Chris Coons (D-Del.), Marsha Blackburn (R-Tenn.), Thom Tillis (R-N.C.) and Amy Klobuchar (D-Minn.) have drafted legislation last October called “Nurture Originals, Foster Art, and Keep Entertainment Safe Act, or NO FAKES.” 

The law would enable victims to sue deepfake creators plus platforms that distribute the material, though difficulties in forensic identification of the offending software plus Section 230 of the Communications Decency Act might make such cases difficult to impossible.

And in January 2024, US Representative Joe Morelle (NY) and Rep. Tom Kean (R-NJ) announced HR 3106, the Preventing Deepfakes of Intimate Images Act, a bill banning the sharing of deepfake pornography. 

But regulations can be frustratingly ineffective. Scientific American points out that ideas such as digital watermarks and other technological safeguards can still be hacked by “sophisticated users.” The same article also points out the limitations of corporate social responsibility and self-regulation. 

Our images, ourselves

Consent counts, but violations of digital consent seem to have run amok, making even the most enthusiastic sex tech proponent think twice about the implications and ethics, say, of AI-generated chatbot images and the creation of AR/VR sexual avatars. 

As Graphika’s “A Revealing Picture” points out, “the increasing prominence and accessibility of these services [AI diffusion models] will very likely lead to further instances of online harm, such as the creation and dissemination of non-consensual nude images, targeted harassment campaigns, sextortion, and the generation of child sexual abuse material.”

So while it might be tempting to create a sexy chatbot or avatar that resembles your favorite celebrity, influencer, or the hottie next door, consent still counts, even in the digital realms. 

So unless you have that express, personal consent from the object of your desire—don’t, just don’t. There are plenty of sexy, imaginary friends you can access through the wonders of technology and even plenty of erotic images you can make, without using images of the unsuspecting and nonconsenting.

Images: Depositphotos

The post Reality Under Siege: Challenges in Detecting and Combating Deepfakes appeared first on Future of Sex.