The online platform X has sparked significant outrage after users utilized its artificial intelligence assistant Grok to change pictures of women by stripping their attire.
Several instances have been seen where the system has been used to remove clothing from women to show them in bikinis without consent, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company behind Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A government official stated that new laws is being brought forward to prohibit nudification software. Under a upcoming criminal offence, anyone who provided such tools would "receive a jail term and heavy fines".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some premium features—which responds to X users' queries when they reference it in a post.
It is commonly used to provide reactions or additional background to other posters' remarks, but people on X are also can edit an posted image through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been condemned for enabling users to generate pictures and footage with nudity and sexualised content, and it was previously accused of creating graphic content of public figures.
"X has been enabling the generation and spread of these images for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "appropriate steps" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it promptly when they learn of it.
