Elon Musk's social media platform X has sparked major anger after users utilized its AI chatbot Grok to manipulate pictures of women by taking off their clothing.
Multiple cases have emerged where the system has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, offered no comment to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to prohibit nudification software. Under a new law, anyone who distributed such technology would "face a jail term and heavy fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' requests when they mention it in a tweet.
It is frequently used to offer reactions or additional context to other posters' comments, but people on X are also able to edit an uploaded photo through its AI image editing feature.
Background of Criticism and Policy
The tool has been criticised for enabling users to produce photos and clips with explicit content and sexually suggestive material, and it was in the past accused of making sexually explicit material of celebrities.
"The platform has been enabling the creation and distribution of these pictures for months without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of people in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate actions" to "lessen the danger" of users viewing unlawful material on their platforms and to remove it promptly when they are notified of it.
