The social media network X has sparked widespread backlash after users employed its artificial intelligence assistant Grok to manipulate pictures of women by removing their clothing.
Multiple examples have surfaced where the system has been used to virtually strip women to render them in bikinis without permission, as well as to insert them in inappropriate situations.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, offered no comment to questions, except for an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to outlaw image stripping technology. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' prompts when they reference it in a post.
It is commonly used to provide commentary or further background to other users' remarks, but people on X are also capable of edit an uploaded photo through its AI image editing feature.
History of Controversy and Rules
The tool has been condemned for permitting users to generate photos and clips with nudity and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been allowing the production and sharing of these images for a long time without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of individuals in a graphic way".
In a statement, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "necessary steps" to "lessen the risk" of users viewing unlawful content on their services and to delete it promptly when they become aware of it.
