The social media platform X has caused widespread backlash after people employed its AI chatbot Grok to change images of women by removing their garments.
Several instances have been seen where the system has been employed to digitally undress women to make them appear in swimwear without permission, as well as to put them in inappropriate situations.
Developer Response and Legal Position
XAI, the firm behind Grok, remained silent to inquiries, except for an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification technology. Under a new criminal offence, anyone who supplied such tools would "receive a jail term and significant fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the potential" of people viewing unlawful material on their services, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some premium features—which answers X users' queries when they tag it in a message.
It is often used to provide reactions or further background to other posters' remarks, but people on X are also capable of modify an posted picture through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been criticised for enabling users to produce pictures and footage with explicit content and sexualised content, and it was previously accused of making graphic material of public figures.
"X has been enabling the creation and spread of these pictures for months without taking any steps and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules bans "showing images of people in a pornographic way".
In a statement, Ofcom said it was illegal to "make or distribute private pictures or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the risk" of users coming across unlawful content on their platforms and to delete it swiftly when they learn of it.
