The online network X has caused significant outrage after people used its AI chatbot Grok to alter photos of women by taking off their garments.
Numerous examples have emerged where the technology has been employed to remove clothing from female subjects to show them in swimwear without permission, as well as to put them in sexualized situations.
Company Response and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, except for an standard message stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to prohibit image stripping technology. Under a new law, anyone who supplied such tools would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their platforms, but did not confirm whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional features—which reacts to X users' requests when they reference it in a post.
It is frequently used to give reactions or additional context to other posters' comments, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been criticised for enabling users to generate images and footage with explicit content and sexually suggestive content, and it was in the past accused of creating sexually explicit material of public figures.
"The platform has been allowing the production and spread of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own usage rules bans "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said platforms such as X were required to take "necessary steps" to "minimize the danger" of users viewing unlawful content on their platforms and to take down it promptly when they learn of it.
