OpenAI, the developer of ChatGPT, has recently released draft documentation discussing the potential for users to generate pornographic and explicit content using its AI technology. This decision has sparked discussions within the tech industry, as OpenAI outlined its intentions for AI products in its Model Spec document. One significant revelation is the company’s exploration of allowing the creation of sexually explicit content in age-appropriate contexts. This comes at a time when AI-generated pornography, also known as “deepfake porn,” is a major concern.
Currently, OpenAI’s usage policies strictly prohibit sexually explicit or suggestive material. However, a note in the Model Spec suggests that the company is considering allowing such content generation. OpenAI is exploring the possibility of responsibly providing the ability to generate NSFW (not safe for work) content through the API and ChatGPT.
Despite OpenAI’s policies, Truth Voices previously reported on AI Girlfriend bots flooding OpenAI’s GPT store, which goes against the rules. The Model Spec document outlines various types of content that could be considered NSFW, including erotica, extreme violence, slurs, and profanity. It remains unclear how far OpenAI’s exploration of explicit content generation will go.
While OpenAI spokesperson Niko Felix claims that the company has no intention of producing AI porn, others involved in writing the Model Spec acknowledge that users may still create sexual content using OpenAI’s technology. The potential relaxation of restrictions on explicit content raises concerns about OpenAI’s ability to moderate such content effectively, especially in light of previous incidents involving AI tools generating explicit imagery without consent.
As OpenAI grapples with the challenges of porn and explicit content generation, the question remains whether the company can prevent malicious actors from misusing their technology to create harmful material. The impacts of AI-generated pornography, particularly on women and girls, have been damaging, leading to harassment and non-consensual imagery circulating online. This raises important questions about content moderation and consent in the development and use of AI technology.
