It looks like Elon Musk’s X could be in hot water again, as it continues to skirt regulations around potentially offensive content.
Over the last few weeks, a new trend has emerged on X, where X users prompt X’s Grok chatbot in-stream to convert images of people, mostly women, into naked or near-naked depictions of the same. That’s also veered into CSAM territory, with some users prompting Elon’s “unhinged” chatbot to strip down images of child actors to bikinis, or worse.
Which has now gotten the attention of regulators in various regions, who are calling on X to take immediate action to restrict the use of its AI tools to create non-consensual, and potentially illegal, visuals.
As reported by TechCrunch, on Friday, India’s IT ministry ordered X to take “corrective action” on Grok, including restricting the generation of content involving “nudity, sexualization, sexually explicit, or otherwise unlawful” material. The ministry gave X 72 hours to detail the steps that it plans to take to prevent the hosting or dissemination of content deemed “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
EU region officials have also raised the alarm, along with regulators in Malaysia, while Britain’s Ofcom has also issued a request to X to outline what steps it’s taking “to comply with their legal duties to protect users in the U.K.”
And that’s likely only the beginning, with several other regions considering new pushes to restrict or ban the app, based on its potential to not only host, but to create such material, violating many local laws.
X owner Elon Musk has acknowledged the concern, and has said that:
“Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
But that may not be enough assurance. And with X already under investigation over Grok’s capacity to create other offensive images (including depictions of well-known characters in Nazi attire), and the U.S. Justice Department recently making a push to address such misuse, it could mean that X will have to implement more restrictions on Grok generations, despite Elon and Co. preferring to take a more hands-off approach to such wherever they can.
At the same time, sexualized content is clearly a part of Elon’s expanded business model for the app.
X’s AI companions, for example, have been specifically built to be able to “engage in NSFW chats” with users.
Which is another element that regulators have been assessing, as they look to get a better understanding of the broader impacts of such, and ensure their safe use, especially by younger users.
And while X would prefer not to censor anything, there are clearly major risk vectors within these new tools, that X doesn’t seem to have fully assessed prior to launch. Which is also in line with Elon’s approach, in moving fast, and expediting product roll-outs, in order to maximize development speed.
The risk, then, is exactly what we’re seeing, with potentially harmful tools released to the public, with minimal guardrails, expanding harm to both users and non-users.
So what will come of this latest push? Well, X will likely put some restrictions in place on some of the most common commands of this type, while it will also seek to outline more stringent measures to restrict underage users from engaging with its AI companions.
But with Elon also looking to raise awareness of his latest AI tools, with a view to maximizing the monetization potential of such (note: Elon also just announced a new update to Grok’s image and video generation capabilities), I don’t expect that X will be looking to scale back its AI generation tools, or implement any stringent limits on what users can create.
Because more people using Grok, for whatever purpose, looks better for xAI, and again, Elon clearly sees sexualized content as a part of X’s expansion plans.
We’ll wait and see if X actually looks to take legal action against the creators of such, as Musk says, but Elon could be headed for another clash with foreign regulators.