🔗 Share this article

The social media network X has caused widespread anger after users used its AI chatbot Grok to alter images of females by taking off their attire.

Numerous instances have been seen where the technology has been employed to digitally undress women to render them in swimwear without permission, as well as to put them in sexualized situations.

Developer Response and Regulatory Stance

XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".

A Home Office representative stated that new laws is being proposed to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such tools would "face a jail term and heavy fines".

"The platform could prevent these types of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."

Regulator Position and Platform Capabilities

The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in regarding artificial intelligence images.

Grok is a free chatbot—with some additional functions—which answers X users' queries when they reference it in a post.

It is commonly used to offer commentary or additional background to other users' comments, but people on X are also capable of edit an uploaded picture through its AI image editing function.

History of Criticism and Rules

The tool has been criticised for allowing users to produce photos and clips with nudity and sexualised material, and it was earlier accused of making sexually explicit content of public figures.

"The platform has been allowing the generation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own terms of service prohibits "depicting likenesses of people in a sexually explicit manner".

In a statement, Ofcom said it was against the law to "generate or share non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.

It said platforms such as X were obligated to take "necessary actions" to "reduce the risk" of users coming across unlawful content on their services and to take down it quickly when they are notified of it.