When Innovation Becomes Exploitation: The Challenges of AI-Generated Child Pornography.

By Brooklyn Nall Hutchins

I. Introduction

The rapid advancement of generative technologies has forced courts, lawmakers, and scholars to reconsider how long-standing doctrines apply in entirely new and unforeseen contexts. The emergence of artificial intelligence (“AI”) has introduced unprecedented capabilities in enabling the creation of hyper-realistic depictions of children in sexually explicit scenarios without involving any real child. Because of this, AI-generated images of child exploitation will impact legal systems and moral boundaries in a greatly negative way. AI-generated child sexual abuse material (“AIG-CSAM”) poses great legal, ethical, and doctrinal challenges. While most child pornography laws hinge on the actual exploitation of minors, AIG-CSAM tests the boundaries of such protections.^[1] Accordingly, legal scholars must take great care in evaluating the risks, legal frameworks, doctrinal challenges, ethics, and policy considerations of AIG-CSAM. Ultimately, the emergence of AIG-CSAM compels an urgent reevaluation of whether existing legal doctrines are capable of addressing the harms posed by synthetic child exploitation material.

II. Negative Capabilities and Risks

Before legal systems can respond effectively, it is necessary to understand the unique dangers that generative AI introduces into the realm of child exploitation. Society must carefully evaluate the negative capabilities and risks of AIG-CSAM. Generative AI models such as Stable Diffusion, Midjourney, and “nudifers” can now produce realistic images or videos of children in sexual scenarios in mere seconds with a single prompt.^[2] Often, such synthetic images or videos are completely indistinguishable from other real photographs.^[3] The sophistication of these systems means that distinguishing whether an image actually depicts a real child or an AI-fabrication of a child is impracticable for legal purposes without significant expertise in how these models work.^[4]

The accessibility of these “nudifying” tools further complicates the problem of AIG-CSAM. Many nudifying platforms are available free of charge, require no advanced technical skill, and can be operated anonymously, lowering barriers to entry for potential offenders.^[5] Unlike other child sexual abuse material (“CSAM”), which requires the abuse of a real child, AIG-CSAM is synthetic. In other words, it may lull potential child sexual abuse perpetrators into a sense of false morality, but the realism of AIG-CSAM can perpetuate harmful fantasies, normalize abuse, and be used as a tool to groom or coerce minors into real abuse.^[6] In fact, studies show that one in ten minors said they or someone they know have used generative AI to produce inappropriate images of other children their age.^[7]

Moreover, the psychological harm may be profound for real individuals whose likenesses are digitally manipulated to create synthetic sexual content. One scholar argues that this type of nonconsensual manipulation invades an individual’s sexual privacy, undermines dignity, and compounds trauma.^[8] The capabilities and risks of generative AI in producing AIG-CSAM lends to the challenges of evaluating this material in a legal context. These risks illustrate that even without direct physical abuse, AIG-CSAM creates an environment where harm can destabilize legal, psychological, and moral boundaries.

III. Legal Framework

The challenges of AIG-CSAM's regulation are most apparent when examining statutory frameworks and case law precedents. Under federal law, child pornography is defined as a visual depiction of sexually explicit conduct involving an actual minor.^[9] In Ashcroft v. Free Speech Coalition, the Supreme Court struck down provisions of the Child Pornography Prevention Act (“CPPA”) that attempted to prohibit depictions that only “appeared to be minors” engaged in sexually explicit conduct.^[10] The Court reasoned that because such material did not involve the exploitation of real children, it had no justification in limiting the First Amendment freedom of speech.^[11]

Importantly, the ruling in Ashcroft creates a doctrinal gap, because CSAM that does not depict actual minors may enjoy constitutional protection, even if it is indistinguishable from real abuse.^[12] Congress has attempted to respond with the Prosecutorial Remedies and Other Tools to end the Exploitation of Children Today Act (“PROTECT”) Act of 2003, which criminalized explicit images or material that is or purports to be sexually explicit material of minors, or that are intended to cause another person to believe the images or videos depict a real child.^[13] More specifically, this act states that “any material or purported material in a manner that reflects the belief, or is intended to cause another to believe, that the material or purported material is, or contains an obscene visual depiction of a minor engaging in sexually explicit conduct,” for public use is not protected.^[14]  Still, this language may be narrowly interpreted and require an extremely high degree of realism before the material is deemed unprotected.^[15] For example, a Maryland appellate court held that to prohibit AI-generated child pornography, a computer generated image that is indistinguishable from an actual child must depicts a real child whose identity can be ascertained.^[16]

Other jurisdictions and countries have taken entirely different approaches. For example, the United Kingdom’s Online Safety Act criminalizes AI-generated images that depict child sexual abuse, whether or not a real child is depicted. The act instead focuses on the material’s high potentional to incite abuse or expose children to harmful imagery.^[17] Similarly, Canada’s Criminal Code prohibits “any photographic, film, video, or other visual representation, whether or not it was made by electronic or mechanical means, that shows a person who is or depicted as being under the age of eighteen years and is engaged in explicitly sexual activity.”^[18] While these varied legal frameworks reflect differing cultural and constitutional priorities, they collectively show the unsettled and fragmented nature of law surrounding AIG-CSAM.

IV. Doctrinal Challenges

Beyond statutory gaps, AIG-CSAM exposes deeper doctrinal challenges that reveal the limits of constitutional interpretation in the new digital era. AIG-CSAM raises fundamental questions about how existing doctrines should be applied, interpreted, or even changed.

One possible avenue of prosecution is obscenity laws. Obscenity, however, is subject to a three-prong test established in Miller v. California^[19]. This three-prong test qualifies material as obscene only (1) if the average person, applying contemporary community standards, would find the work appeals to the prurient interest, (2) if the work depicts conduct in a patently offensive way, and (3) if the work or material lacks other serious literary, artistic, political, or scientific value.^[20]

Unlike CSAM, which is categorically excluded from First Amendment protection, obscenity is a much narrower exception, heavily dependent on local community standards and evaluative judgments, which AIG-CSAM may or may not meet.^[21] Synthetic child sexual abuse material may evade prosecution, unless it squarely meets the Miller test.^[22] As a result, federal prosecutors may hesitate to pursue charges, given the constitutional uncertainties and high evidentiary burden Miller calls for^[23]. Another issue with using obscenity laws to prosecute child pornography is the ambiguity of interpretation. For example, in Stanley v. Georgia, the court held that mere private possession of obscene materials is not a crime and is protected by the First Amendment^[24]. On the other hand, in Osborne v. Ohio, the court held that child pornography is not protected by the First Amendment and can be criminalized, even if only held privately^[25].

The First Amendment looms over AIG-CSAM largely in this discussion. In Ashcroft, the Court emphasized that the government could not suppress lawful speech merely because it might encourage lawful behavior.^[26] Yet, critics argue that this reasoning fails to account for the evolving nature of digital harm, particularly when content can be used to create new abusers, groom children, or retraumatize survivors.^[27]

When AIG-CSAM is created using real children’s likenesses, often scraped from social media or altered from otherwise innocent photographs, the harm becomes immediate and personal. Courts have begun to recognize the tort of nonconsensual pornography, but statutory remedies remain limited.^[28] Expanding tort doctrines or statutory remedies may be helpful in some scenarios, but these measures do not address the criminal law gap left by Ashcroft and Miller^[29]. These doctrinal challenges highlight that without significant legal innovation, AIG-CSAM will continue to fall short of obscenity law, free speech protections, and tort remedies.

V. Ethical and Policy Considerations

The difficulty of resolving AIG-CSAM issues also lies in the ethical and policy considerations surrounding platform’s responsibility and child protection. Generative AI platforms often disclaim responsibility for user-generated content under safe-harbor provisions, but many scholars argue that platforms should bear some responsibility for foreseeable misuse, especially when tools are designed or easily adapted to generate explicit content.^[30]

Even when no real child is depicted, AIG-CSAM can desensitize offenders, embolden harmful behavior, and retraumatize survivors whose images are manipulated^[31]. Ethical frameworks must consider not only tangible harms, but also possible symbolic and future harms, such as the violence of synthetic exploitations and the deterioration and erosion of dignity and sexual consent. Policymakers must confront whether protecting children in the digital age requires broad prophylactic rules that preemptively limit AIG-CSAM, or whether narrower, harm-based approaches suffice.  These ethical and policy considerations reveal that addressing AIG-CSAM requires proactive anticipation of technological misuse^[32].

VI. Conclusion

AI-generated images or videos of child sexual abuse represent a fundamentally new form of exploitation. It is synthetic, technologically enabled, and legally elusive. The gaps exposed by generative tools demand urgent legislative and ethical responses. Protecting children in the modern, digital age requires more than prosecuting “traditional” forms of abuse; it requires confronting the symbolic and future harms of synthetic sexual abuse depictions, the doctrinal shortcomings of current federal law, and the moral responsibility of AI-platforms.

AIG-CSAM may not involve a camera or a child victim in a traditional sense, but its impact is seen across legal systems, moral boundaries, and the trust society places in technology. Without rapid legislative action, the legacy of AI will not be one of innovation, but of abuse and exploitation.

^[1] See 18 U.S.C. § 2256(8); see Ashcroft v. Free Speech Coal., 535 U.S. 234, 234 (2002).

^[2] AI-generated Child Sexual Abuse: The New Digital Threat We Must Confront Now, Thorn (Aug. 13, 2025), https://www.thorn.org/blog/ai-generated-child-sexual-abuse-the-new-digital-threat-we-must-confront-now/.

^[3] Id.

^[4] See id.

^[5] See id; Riana Pfefferkorn, , Addressing AI-Generated Child Sexual Abuse Material: Opportunities for Educational Policy, Standford Univ. Human-Centered Artificial Intelligence (July 21, 2025), https://hai.stanford.edu/policy/addressing-ai-generated-child-sexual-abuse-material-opportunities-for-educational-policy.

^[6] Laura Jayne Broom et al.,, A Systematic Review of Fantasy Driven vs. Contact Driven Internet-Initiated Sexual Offenses: Discrete or Overlapping Typologies?, 79 Child Abuse & Neglect (May 2018), https://www.sciencedirect.com/science/article/abs/pii/S0145213418300851.

^[7] Id.

^[8] Danielle Keats Citron, Sexual Privacy, 128 Yale L.J. 1870 (2019).

^[9] 18 U.S.C. § 2256.

^[10] 535 U.S. 234, 256 (2002).

^[11] Id. at 250–51 (holding the images that only appeared to be children recorded no real crime and created no real victims).

^[12] Id. at 258.

^[13] 18 U.S.C. § 2252A(a)(3)(B)(i)–(ii).

^[14] Id.

^[15] See Brasse v. State, 264 Md.App. 740, 761 (2025).   

^[16] Id.

^[17]Department for Science, Innovation and Technology, Online Safety Act: Explainer, Gov UK (Apr. 24, 2025),  https://www.gov.uk/government/publications/online-safety-act-explainer.

^[18] Criminal Code, R.S.C. 1985, c. C-46, § 163.1.

^[19] 413 U.S. 15, 24 (1973).

^[20] Id.

^[21] See id.

^[22] See id.

^[23] See id.

^[24] Stanley v. Ga., 394 U.S. 557, 565 (1969) (“Whatever may be the justifications for other statutes regulating obscenity, we do not think they reach into the privacy of one’s home”).

^[25] Osborne v. Ohio, 495 U.S. 103, 112 (1990).

^[26] Ashcroft v. Free Speech Coal., 535 U.S. 234, 253 (2002).

^[27] Internet Watch Foundation, AIG-CSAM REPORT UPDATE, What has changed in the AI CSAM landscape?, (July 2024), chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.iwf.org.uk/media/nadlcb1z/iwf-ai-csam-report_update-public-jul24v13.pdf.

^[28] Citron, supra note 8, at 1870.

^[29] See Miller, 413 U.S. at 24; see Ashcroft, 353 U.S. at 253.

^[30] Navigating Legal Risks with AIGenerated Content, Bloomberg Law (March 2023), 3, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.ropesgray.com/-/media/files/news/2023/04/artificial-intelligence-generated-content-legal-risk-navigation-bloomberg-law-article.pdf?rev=7a2166a85d254a338db064b4dc68007e&hash=F4178A43C5BA2A26A3.

^[31] See AI-generated Child Sexual Abuse: The New Digital Threat We Must Confront Now, Thorn (Aug. 13, 2025), https://www.thorn.org/blog/ai-generated-child-sexual-abuse-the-new-digital-threat-we-must-confront-now/.

^[32] See id.