The social media network X has provoked widespread outrage after individuals used its artificial intelligence assistant Grok to alter photos of women by stripping their attire.
Multiple cases have emerged where the system has been employed to digitally undress female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized contexts.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being introduced to ban image stripping technology. Under a proposed law, anyone who supplied such technology would "receive a jail term and significant penalties".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium functions—which reacts to X users' requests when they reference it in a message.
It is frequently used to give commentary or additional background to other posters' comments, but people on X are also capable of modify an posted image through its artificial intelligence picture modification function.
History of Criticism and Rules
The chatbot has been faulted for enabling users to produce pictures and footage with explicit content and sexualised content, and it was previously accused of creating graphic material of celebrities.
"X has been permitting the generation and sharing of these images for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of people in a graphic way".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said platforms such as X were mandated to take "appropriate actions" to "minimize the danger" of users coming across unlawful content on their platforms and to delete it quickly when they become aware of it.
