The social media platform X has provoked major anger after users used its artificial intelligence assistant Grok to manipulate images of women by taking off their clothing.
Numerous instances have emerged where the tool has been used to remove clothing from women to show them in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to inquiries, aside from an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to prohibit nudification software. Under a upcoming criminal offence, anyone who provided such tools would "face a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for features—which responds to X users' prompts when they mention it in a tweet.
It is often used to offer reactions or more context to other users' comments, but people on X are also able to alter an posted image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The tool has been criticised for enabling users to create images and footage with nudity and sexualised content, and it was previously accused of creating sexually explicit content of celebrities.
"The platform has been allowing the generation and spread of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of people in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and confirmed this included explicit AI images created with AI.
It said platforms such as X were obligated to take "necessary measures" to "lessen the risk" of users encountering illegal content on their platforms and to remove it promptly when they learn of it.
