The move comes after Grok AI produced sexualized images that raised concerns among regulators worldwide. Authorities from Europe to Asia have expressed alarm over sexually explicit content generated by AI systems.
Governments and regulators are increasingly cracking down on such content. Many are imposing bans or demanding safeguards to prevent the spread of illegal or harmful material through AI tools.
xAI’s decision to restrict image editing is part of a broader global effort to make AI technology safer and more responsible. The company aims to address regulatory concerns while continuing to provide AI services.
Read more: OpenAI signs $10 billion deal with Cerebras
The restrictions show growing attention on AI-generated content and the need for clear safety measures, especially as AI tools become more advanced and widely used.
xAI acted quickly to prevent further controversy. The move highlights global concern over AI-generated sexual content. Regulators are now closely monitoring AI tools. AI companies may face stricter rules as technology evolves.
"We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the company said in an X post.
"This restriction applies to all users, including paid subscribers."
Hyper-realistic images of women manipulated to look like they were in microscopic bikinis, in degrading poses or covered in bruises began flooding social media platform X this month. In some cases, minors were digitally stripped down to swimwear, sparking broad criticism.
Last week, Grok began allowing only paying subscribers to use its image generation and editing features.
X curtailed Grok's ability to generate or edit images publicly for many of its users, but the chatbot still privately produced sexually charged images on demand on Wednesday before xAI's announcement, Reuters found.
Read more: Spotify opens new doors for creators after $10 billion podcast push
Billionaire Musk owns xAI, which in turn owns X, formerly known as Twitter.
On Wednesday, xAI added that it blocks users, based on their location, from generating images of people in skimpy attire in "jurisdictions where it's illegal". It did not identify those jurisdictions.
CALIFORNIA OFFICIALS DEMAND ANSWERS
Earlier on Wednesday, California's governor and attorney general said they were demanding answers from xAI after Musk said he was not aware of any "naked underage images" generated by Grok.
"We’re demanding immediate answers from xAI on their plan to stop the creation & spread of this content," California Attorney General Rob Bonta wrote on X.
Governor Gavin Newsom called on Bonta "to immediately investigate the company and hold xAI accountable."
The comments by Newsom and Bonta were the most serious so far by U.S. officials tackling the explosion of AI-generated nonconsensual sexualized imagery on X.
The California move added to the pressure Musk faces in the United States and around the world. Lawmakers and advocacy groups have called for Apple (AAPL.O), opens new tab and Google (GOOGL.O), opens new tab to drop Grok from app stores.
At first, Musk publicly laughed off the controversy, posting humorous emojis in response to users' comments about the influx of sexualized photos.
More recently, X has said it treats reports of child sexual abuse material seriously and polices it vigorously.
Earlier on Wednesday, Musk said he was "not aware of any naked underage images generated by Grok. Literally zero."
X did not immediately respond to questions about the California announcement and Musk's comments.
xAI did not respond directly to an emailed request for comment on California officials' statements or Musk's post that he was unaware of sexualized imagery of minors.
