The online network X has sparked widespread outrage after people utilized its AI chatbot Grok to manipulate images of women by stripping their clothing.
Numerous instances have surfaced where the system has been used to remove clothing from female subjects to show them in bikinis without consent, as well as to put them in inappropriate scenarios.
Company Response and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, aside from an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw image stripping software. Under a proposed law, anyone who distributed such technology would "receive a jail term and heavy fines".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "consider the danger" of people viewing unlawful material on their platforms, but did not specify whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which answers X users' queries when they mention it in a message.
It is frequently used to give commentary or additional context to other users' remarks, but people on X are also able to edit an uploaded picture through its AI image editing tool.
Background of Criticism and Policy
The tool has been criticised for permitting users to generate pictures and footage with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate content of celebrities.
"X has been permitting the generation and distribution of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting images of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or share non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the risk" of users coming across unlawful content on their services and to delete it quickly when they learn of it.
