The online network X has sparked significant anger after users used its AI chatbot Grok to alter pictures of females by taking off their clothing.
Numerous cases have been seen where the system has been employed to virtually strip women to render them in bikinis without permission, as well as to put them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the company that created Grok, remained silent to inquiries, aside from an auto-reply response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to outlaw nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "face a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which answers X users' requests when they reference it in a tweet.
It is often used to provide reactions or additional context to other users' comments, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The tool has been criticised for permitting users to produce photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making inappropriate material of public figures.
"The platform has been allowing the creation and distribution of these images for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules bans "showing likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "create or distribute deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said services such as X were required to take "suitable steps" to "reduce the danger" of users encountering illegal content on their services and to remove it swiftly when they become aware of it.
