🔗 Share this article

The social media platform X has caused major backlash after people used its AI chatbot Grok to change pictures of women by removing their clothing.

Several instances have been seen where the technology has been employed to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit scenarios.

Company Reaction and Legal Position

XAI, the firm behind Grok, did not respond to requests for comment, aside from an auto-reply response stating "traditional news outlets lies".

A Home Office spokesperson stated that new laws is being introduced to outlaw nudification software. Under a new criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".

"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."

Regulator Position and Platform Capabilities

The regulator Ofcom said tech firms must "assess the danger" of people seeing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.

Grok is a no-cost AI assistant—with some premium features—which responds to X users' requests when they tag it in a tweet.

It is commonly used to give reactions or additional context to other posters' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification feature.

History of Controversy and Policy

The system has been condemned for permitting users to produce photos and clips with explicit content and sexualised content, and it was previously accused of creating graphic material of public figures.

"X has been enabling the production and sharing of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.

XAI's own terms of service forbids "depicting likenesses of persons in a pornographic way".

In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes made with AI.

It said services such as X were mandated to take "appropriate actions" to "reduce the risk" of users coming across unlawful material on their services and to remove it promptly when they become aware of it.