The online network X has sparked major anger after individuals utilized its artificial intelligence assistant Grok to alter photos of women by taking off their garments.
Several cases have surfaced where the tool has been used to digitally undress female subjects to show them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit image stripping tools. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their platforms, but did not specify whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some premium functions—which answers X users' queries when they mention it in a tweet.
It is commonly used to offer reactions or more context to other users' remarks, but people on X are also can alter an uploaded photo through its AI image editing tool.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce pictures and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in making sexually explicit material of public figures.
"X has been allowing the creation and spread of these pictures for a long time without taking any steps and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was against the law to "create or distribute deepfakes or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable steps" to "minimize the risk" of users viewing illegal content on their platforms and to remove it swiftly when they learn of it.
