OpenAI’s Sora App Grapples with Unsettling Niche Fetish Videos

OpenAI’s innovative video-making application, Sora 2, designed to revolutionize content creation, is facing unexpected challenges related to content moderation and user ethics. While celebrated for its advanced capabilities, a concerning trend has emerged where users are creating niche fetish videos, often featuring the likenesses of real individuals without explicit consent for such uses.

A Journalist’s Unsettling Discovery

Senior technology and culture correspondent, Katie Notopoulos, shared her personal experience after opting to make her likeness available for “cameos” on Sora. Initially viewing it as a harmless feature, Notopoulos soon discovered videos of herself crafted by strangers depicting scenarios like exaggerated pregnancy or “belly inflation.” A deeper dive into these creators’ profiles revealed entire collections dedicated to such specific niche fetishes.

The Pervasive Nature of Niche Content

What began as isolated incidents quickly escalated. Notopoulos observed that a significant portion of popular cameos featuring her face, as well as those of other female creators, tech reporters, and venture capital employees, now included various fetish content—from centaur transformations to cloning scenarios. This phenomenon extends beyond real-person cameos, with numerous accounts dedicated to generating similar content using purely AI-created women, covering themes like vore, foot worship, and crushing.

Navigating Consent and Digital Ethics

While acknowledging a long history of niche content on the internet, Notopoulos highlights a critical distinction with Sora: the effortless ability to generate potentially sexually gratifying content using a real person’s face without their specific consent for that particular use. This raises significant ethical questions regarding digital likeness and boundaries, especially for individuals who might be less accustomed to the internet’s “underbelly.”

OpenAI has implemented measures against overt nudity or sexual content and has also paused the use of deceased celebrity likenesses following complaints. However, the surge in non-nude, quasi-sexual fetish content presents a new, unforeseen challenge for the platform’s moderation policies.

Broader Safety Concerns and Minors

The issues extend beyond niche fetishes. Concerns have also surfaced regarding the interaction of minors with adult content creators on Sora. Notopoulos cited examples of young users creating videos with prominent porn stars, and an adult user generating multiple videos of a preteen girl dancing. While OpenAI has announced parental controls for teen accounts, clarity on rules regarding these grey-area interactions remains elusive.

The Road Ahead for OpenAI

OpenAI, which did not respond to requests for comment, appears to be navigating uncharted territory with its rapidly adopted AI video app. The company’s general approach, often characterized as “asking forgiveness, not permission” in matters of copyright and intellectual property, now faces a crucial test in defining its stance on user-generated fetish content—both involving real people and purely AI-generated characters.

The emerging landscape suggests a chilling effect on women’s willingness to engage publicly with such AI tools. The ease with which “creepy” content can be generated using real faces underscores an inherent risk that many women instinctively recognize, prompting a broader conversation about safety, consent, and responsible AI development in public-facing applications.