The online network X has sparked major outrage after users employed its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Multiple instances have been seen where the system has been used to virtually strip female subjects to show them in swimwear without consent, as well as to place them in sexualized contexts.
Company Response and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, except for an standard response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to ban nudification tools. Under a upcoming law, anyone who supplied such tools would "face a jail term and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people viewing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' requests when they mention it in a tweet.
It is often used to give commentary or additional context to other posters' comments, but people on X are also able to alter an posted image through its AI image editing tool.
Background of Criticism and Policy
The chatbot has been condemned for allowing users to create photos and footage with nudity and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit material of celebrities.
"X has been enabling the production and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service bans "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or circulate private pictures or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the danger" of users viewing unlawful material on their platforms and to take down it swiftly when they become aware of it.
