eSafety Commissioner demands crackdown on Musk’s AI ‘nudify’ images on X

16 hours ago 4

Elon Musk’s social media platform X is being investigated by Australia’s eSafety Commissioner over a surge in sexually explicit images generated by its AI chatbot, following a threat from UK Prime Minister Keir Starmer to ban X over the same issue.

On Friday, commissioner Julie Inman Grant said her agency could use its existing regulatory powers to “investigate and take appropriate action” against AI products that fail to provide appropriate safeguards, as evidence mounts that the controversial platform is being used to generate child abuse material.

“I’m deeply concerned about the increasing use of generative AI to sexualise or exploit people, particularly where children are involved,” Inman Grant said in a statement on LinkedIn.

Inman Grant said reports of sexualised images generated by X’s artificial intelligence tool Grok had doubled since late last year, relating to adults and children, and that companies had the capability to prevent misuse of their products.

“We’ve now entered an age where companies must ensure generative AI products have appropriate safeguards and guardrails built in across every stage of the product lifecycle,” she said.

“By adopting a range of well-established AI safety practices, they can effectively anticipate how bad actors will exploit design features and loopholes before harm occurs.”

X, formerly known as Twitter, has published many thousands of images of people that have been depicted naked by AI without their consent.

Starmer on Thursday described the images as unlawful and intolerable and said his government would take action, including via Ofcom, the British communications regulator.

“This is disgraceful, it’s disgusting, and it’s not to be tolerated. X has got to get a grip of this,” Starmer said.

“Ofcom has our full support to take action in relation to this.”

The UK regulator said earlier this week that it was investigating allegations of online child sexual-abuse material on the dark web allegedly generated by Grok, and had made contact with Musk’s company over the reports.

The eSafety agency said in a statement on Friday that it had received several reports in the past couple of weeks about the use of Grok to generate sexualised or exploitative imagery, and that it had contacted X to find out what safeguards it was putting in place to prevent misuse.

A spokesman for the Albanese government said that use of generative AI to produce sexualised material without consent was “abhorrent” and it was committed to restricting access to nudify tools in Australia.

New codes coming into force on March 9 will compel AI services, among others, to limit children’s access to sexually explicit content, as well as violent material and themes related to self-harm and suicide.

These regulations follow the eSafety commission’s enforcement action against a UK-based company that offered two widely used nudify services that allowed its users to manipulate photos of real people.

These services had attracted about 100,000 visitors a month in Australia, and eSafety said they had been used to generate explicit deepfake images of students in Australian schools. The sites were withdrawn from Australia in November after an official warning from eSafety.

One month since eSafety instituted the Albanese government’s world-first under-16 social media ban, Inman Grant is still under threat of contempt charges by US Congress unless she testifies about online safety laws by Tuesday, January 13.

Republican Congressman Jim Jordan, who chairs the House Judiciary Committee, has claimed that Australia’s Online Safety Act threatens the free speech of American citizens, and has twice demanded Inman Grant to account for her previous attempts to force social media companies to remove graphic content from their platforms.

Congress has the power to compel US citizens living abroad to testify, and to be found in contempt of Congress for failing to comply. Contempt of Congress can be punished with fines or jail time.

Inman Grant butted heads with Musk last year when eSafety demanded that X remove graphic footage of a church stabbing in Sydney, arguing that global removal was necessary because Australians could use VPNs to access the content. The case was abandoned.

The European Commission has ordered X to retain all documents relating to Grok for longer while the bloc ensures compliance with its rules after condemning it for producing sexualised images, a spokesperson said on Thursday.

The Italian regulator said it was working with Ireland’s Data Protection Commission – the lead EU privacy authority for X, which has a base in Ireland – and reserved the right to take further action.

Sweden on Thursday joined the chorus of criticism, saying the AI-generated images were unacceptable after the country’s deputy prime minister was targeted by a Grok user’s prompt this week.

with Bloomberg, Reuters

Be the first to know when major news happens. Sign up for breaking news alerts on email or turn on notifications in the app.