Elon Musk's online platform X has caused widespread backlash after individuals used its AI chatbot Grok to manipulate pictures of women by stripping their attire.
Numerous examples have surfaced where the tool has been employed to digitally undress female subjects to render them in swimwear non-consensually, as well as to place them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to questions, except for an standard response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification tools. Under a upcoming law, anyone who supplied such tools would "be subject to a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people accessing illegal content on their services, but did not specify whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which answers X users' queries when they tag it in a message.
It is commonly used to give commentary or further background to other users' comments, but people on X are also can modify an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Policy
The tool has been condemned for enabling users to create photos and clips with nudity and sexualised material, and it was earlier alleged to have been involved in making graphic material of public figures.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of people in a pornographic way".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said networks such as X were required to take "appropriate steps" to "lessen the danger" of users coming across unlawful material on their platforms and to remove it swiftly when they become aware of it.
