🔗 Share this article

The social media platform X has sparked widespread backlash after people employed its AI chatbot Grok to manipulate pictures of women by taking off their clothing.

Numerous cases have emerged where the system has been used to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in sexually explicit contexts.

Company Response and Legal Stance

XAI, the company that created Grok, remained silent to inquiries, except for an auto-reply message stating "traditional news outlets lies".

A government representative stated that legislation is being introduced to outlaw nudification tools. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and heavy fines".

"X or Grok has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."

Oversight Body Stance and Tool Capabilities

The communications watchdog Ofcom said platforms must "assess the potential" of people accessing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.

Grok is a free artificial intelligence—with some premium functions—which responds to X users' queries when they mention it in a message.

It is often used to give commentary or further background to other users' remarks, but people on X are also able to edit an uploaded picture through its AI image editing tool.

History of Controversy and Policy

The chatbot has been faulted for enabling users to produce photos and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating graphic material of celebrities.

"X has been enabling the creation and sharing of these pictures for a long time without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.

XAI's own terms of service prohibits "portraying likenesses of persons in a pornographic way".

In a comment, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.

It said networks such as X were obligated to take "appropriate actions" to "lessen the danger" of users viewing illegal material on their platforms and to take down it promptly when they are notified of it.