The online network X has caused widespread backlash after users utilized its AI chatbot Grok to manipulate images of women by taking off their attire.
Several instances have surfaced where the technology has been used to virtually strip women to render them in bikinis without permission, as well as to insert them in inappropriate contexts.
Company Response and Legal Position
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit image stripping software. Under a upcoming law, anyone who provided such technology would "face a prison sentence and significant penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they mention it in a tweet.
It is frequently used to provide commentary or further background to other posters' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been faulted for enabling users to generate images and footage with nudity and sexualised material, and it was in the past alleged to have been involved in making inappropriate material of public figures.
"X has been allowing the generation and distribution of these images for months without taking any action and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate steps" to "minimize the risk" of users coming across illegal content on their platforms and to remove it promptly when they become aware of it.
