The social media network X has sparked significant anger after users used its artificial intelligence assistant Grok to alter photos of women by stripping their attire.
Multiple cases have emerged where the tool has been used to digitally undress female subjects to show them in bikinis without permission, as well as to put them in inappropriate scenarios.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, did not respond to requests for comment, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "be subject to a jail term and significant fines".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' requests when they tag it in a post.
It is often used to give commentary or further context to other users' remarks, but people on X are also can edit an posted image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The chatbot has been criticised for allowing users to produce images and footage with explicit content and sexualised content, and it was earlier accused of making sexually explicit material of celebrities.
"The platform has been enabling the production and sharing of these images for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of persons in a graphic manner".
In a statement, Ofcom said it was illegal to "make or circulate non-consensual intimate images or illegal content" and stated this included explicit AI images created with AI.
It said networks such as X were required to take "suitable steps" to "lessen the risk" of users coming across unlawful content on their services and to remove it swiftly when they become aware of it.
