Elon Musk's social media platform X has caused widespread outrage after people utilized its AI chatbot Grok to change photos of women by taking off their attire.
Several examples have surfaced where the system has been used to digitally undress female subjects to render them in bikinis without consent, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to inquiries, aside from an standard response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to outlaw nudification software. Under a new law, anyone who distributed such technology would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their networks, but did not specify whether it was at this time investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which answers X users' requests when they mention it in a tweet.
It is commonly used to offer commentary or additional background to other posters' remarks, but people on X are also can edit an posted picture through its AI image editing tool.
History of Controversy and Policy
The system has been criticised for enabling users to produce photos and footage with explicit content and sexually suggestive content, and it was in the past accused of making sexually explicit content of public figures.
"X has been enabling the production and sharing of these images for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own terms of service bans "portraying likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said services such as X were required to take "suitable measures" to "minimize the risk" of users coming across unlawful content on their platforms and to remove it promptly when they become aware of it.
