The Internet Watch Foundation (IWF) says its analysts have discovered “criminal imagery” of girls aged between 11 and 13 which “appears to have been created” using Grok.
The AI tool is owned by Elon Musk’s firm xAI. It can be accessed either through its website and app, or through the social media platform X.
The IWF said it found “sexualised and topless imagery of girls” on a “dark web forum” in which users claimed they used Grok to create the imagery.
The BBC has approached X and xAI for comment.
The IWF’s Ngaire Alexander told the BBC tools like Grok now risked “bringing sexual AI imagery of children into the mainstream”.
He said the material would be classified as Category C under UK law – the lowest severity of criminal material.
But he said the user who uploaded it had then used a different AI tool, not made by xAI, to create a Category A image – the most serious category.
“We are extremely concerned about the ease and speed with which people can apparently generate photo-realistic child sexual abuse material (CSAM),” he said.
The charity, which aims to remove child sexual abuse material from the internet, operates a hotline where suspected CSAM can be reported, and employs analysts who assess the legality and severity of that material.
Its analysts found the material by on the dark web – the images were not found on the social media platform X.
X and xAI were previously contacted by Ofcom, following reports Grok can be used to make “sexualised images of children” and undress women.
The BBC has seen several examples on the social media platform X of people asking the chatbot to alter real images to make women appear in bikinis without their consent, as well as putting them in sexual situations.
The IWF said it had received reports of such images on X, however these had not so far been assessed to have met the legal definition of CSAM.
In a previous statement, X said: “We take action against illegal content on X, including CSAM, by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.
“Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”