Malaysia, Indonesia become first countries to block Musk’s grok over erotic AI images
listen to this article
estimated 4 minutes
The audio version of this article has been generated by AI-based technology. There may be mispronunciations. We are working with our partners to continually review and improve results.
Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk’s company xAI, as concerns grow among global authorities that it is being misused to generate sexually explicit and non-consensual images.
The move reflects growing scrutiny of generative AI tools that can generate realistic images, sounds and text, as well as concerns that existing safeguards are failing to prevent their misuse.
Grok chatbot, which was launched on Musk’s social media platform
Last week, Grok limited image creation and editing to paying users following a global backlash over erotic deepfakes of people, but critics say it didn’t fully solve the problem.
An emailed request for comment to xAI by The Associated Press resulted in an automated reply from the media support email address stating, “Legacy media lies.” The same message was received in a separate email seeking comment regarding the global response.
Regulators in the two Southeast Asian countries said existing controls are not stopping the creation and spread of fake pornographic material, especially involving women and minors. The Indonesian government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday.
“The government views non-consensual sexual relations in the digital sphere as a serious violation of citizens’ human rights, dignity and security,” Indonesian Communications and Digital Affairs Minister Meutya Hafid said in a statement on Saturday.
The ministry said the measure aims to protect women, children and the wider community from fake pornographic content generated using AI.
Preliminary findings showed that Grok lacks effective safeguards to prevent users from creating and distributing pornographic content based on real photographs of Indonesian residents, Alexander Sabar, director general of the Digital Space Supervision, said in a separate statement.
They said such practices risk violating privacy and image rights when photographs are manipulated or shared without consent, causing psychological, social and reputational harm.
In Kuala Lumpur, the Malaysian Communications and Multimedia Commission on Sunday ordered a temporary ban on Grok after it said the tool was “repeatedly misused” to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.
The regulator said the responses to notices issued this month to X Corp and XAI demanding stronger security measures largely relied on user reporting mechanisms.
“The ban has been imposed as a preventive and proportionate measure while the legal and regulatory processes are ongoing,” it said, adding that access will remain blocked until effective security measures are put in place.
Investigation started in Britain
Launching in 2023, Grok is free to use on X. Users can ask questions to it on social media platforms and directly tag created posts or reply to other users’ posts. Last summer, the company added an image generator feature, Grok Imagine, which included a so-called spicy mode that can generate adult content.
The Southeast Asian sanctions come amid growing scrutiny of Grok elsewhere, including the European Union, the United Kingdom, India and France.
A Toronto mom says her 12-year-old son asked Tesla’s Grok which soccer player is better: Cristiano Ronaldo or Lionel Messi. She says that after some back-and-forth, the chatbot asked her son, ‘Why don’t you send me some nude photos?’
On Monday, the UK media regulator said it had launched a formal investigation into whether Grok “fulfilled its duties to protect people in the UK from illegal content.”
Regulator Ofcom said Grok-generated images of children being sexually abused or people being stripped could amount to pornography or child sexual exploitation material.
Technology Secretary Liz Kendall said, “The content created and shared using Grok in recent days is extremely disturbing.”
Canada is not considering banning X, AI Minister Evan Solomon said in a post on the forum last week. His office said discussions about the deepfake controversy are ongoing.
