Hajra Naz
France and Malaysia have joined India in condemning Grok, the AI chatbot developed by Elon Musk’s startup xAI, over its role in generating sexualized deepfakes of women and minors.
The scrutiny comes after reports that Grok was used to create explicit, nonconsensual images, including content involving underage girls. The chatbot is integrated into Musk’s social media platform X, where the photos were reportedly generated and shared.
Grok Issues an Apology—But Questions Remain
Earlier this week, Grok posted a public apology on its X account, stating:
The statement acknowledged that the incident violated ethical standards and may have breached U.S. laws related to child sexual abuse material (CSAM). It added that the failure stemmed from insufficient safeguards and that xAI is reviewing its systems to prevent similar incidents in the future.
However, the apology immediately raised questions about accountability. Defector writer Albert Burneko pointed out that Grok is not a sentient actor capable of responsibility, calling the apology “without substance” and arguing that an AI system cannot meaningfully be held accountable for enabling the creation of CSAM.
Broader Pattern of Abuse Raises Alarm
Investigations suggest the issue extends beyond a single incident. Futurism reported that Grok has also been used to generate nonconsensual pornographic images of adult women, as well as images depicting sexual assault and abuse.
These findings have intensified concerns that generative AI tools, when paired with weak moderation, can rapidly scale harmful and illegal content.
In response, Elon Musk posted on Saturday:
“Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
Critics argue that enforcement after the fact does little to address systemic design flaws that allow such content to be generated in the first place.
India Threatens Legal Consequences
India’s Ministry of Electronics and Information Technology issued a formal order on Friday directing X to restrict Grok from generating content deemed “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
The ministry gave X 72 hours to respond, warning that failure to comply could result in the platform losing its “safe harbor” protections—legal safeguards that shield companies from liability for user-generated content.
Read More: Grok claims Elon Musk is better than everyone, except Shohei Ohtani
France Launches Criminal Investigation
French authorities have also moved quickly. The Paris prosecutor’s office confirmed to Politico that it is investigating the spread of sexually explicit deepfakes on X.
France’s digital affairs office said three government ministers had formally reported what they described as “manifestly illegal content” to both the prosecutor’s office and a government-run online monitoring platform, requesting immediate removal.
Malaysia Investigates AI Misuse on X
Malaysia’s Communications and Multimedia Commission issued its own statement, saying it had taken “serious concern” over public complaints regarding the misuse of AI tools on X.
The commission cited the digital manipulation of images involving women and minors to create “indecent, grossly offensive, and otherwise harmful content,” adding that it is actively investigating the matter.
A Growing Global Test for AI Platforms
The investigations in France, Malaysia, and India point to a wider global reckoning. Generative AI is under growing scrutiny. So is platform responsibility. Existing safeguards are being questioned.
As AI tools become more powerful and easier to use, regulators are losing patience. Apologies are no longer enough. Governments want real technical fixes. They also want legal accountability. That demand is even stronger when minors are harmed.
