🔗 Share this article

The social media network X has provoked significant outrage after people used its AI chatbot Grok to manipulate photos of women by taking off their garments.

Multiple cases have emerged where the technology has been employed to remove clothing from female subjects to make them appear in bikinis without consent, as well as to place them in sexually explicit scenarios.

Company Reaction and Regulatory Position

XAI, the firm behind Grok, offered no comment to questions, other than an automatically-generated message stating "legacy media lies".

A Home Office official stated that new laws is being introduced to ban nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "receive a prison sentence and heavy fines".

"The platform is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."

Oversight Body Position and Platform Functionality

The communications watchdog Ofcom said platforms must "assess the risk" of people accessing illegal content on their services, but did not confirm whether it was at this time investigating X or Grok in regarding artificial intelligence pictures.

Grok is a no-cost AI assistant—with some premium functions—which answers X users' requests when they mention it in a message.

It is commonly used to offer reactions or additional context to other posters' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification function.

Background of Controversy and Rules

The chatbot has been faulted for permitting users to create images and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating graphic material of celebrities.

"X has been allowing the production and spread of these pictures for months without taking any measures and we have not seen any action by regulators," Professor McGlynn added.

XAI's own usage rules prohibits "depicting images of persons in a sexually explicit way".

In a comment, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.

It said services such as X were obligated to take "suitable steps" to "lessen the danger" of users encountering unlawful material on their platforms and to take down it promptly when they become aware of it.