Social media users fall for AI-generated images of children 'engaging in satanic ritual'

"Baphomet book club," reads a Korean-language post shared on Telegram on May 7, 2023.

The post includes 10 images that appear to show children dressed in satanic-themed clothes sitting around a pentagram or alongside adults dressed like demons.

Other posts, including those on the popular South Korean social media platform Naver Blog, claim these images show "children of those who worship Satan, as if to signal the end times marked by the recent 10th anniversary of SatanCon in Boston".

Image

Screenshot of the misleading post shared on Facebook. Captured May 11, 2023.

The images were also shared alongside a similar claim on Facebook and Daum Cafe. They also circulated in posts written in English and Spanish.

Comments left in the posts indicate some users were misled to believe the images showed a genuine event.

One user wrote, "These devilish people are running loose!"

"What's next, a Satanic school?" said another.

However, the images' creator told AFP that they were digitally generated using AI software and do not show an actual event.

Digital creation

The photographs bear a watermark that reads "The Pumpkin Empress". A Google search of this phrase led to the original post from a social media account with the same name that shared the images on May 2, 2023 on Instagram and Facebook (archived here and here).

The Facebook post had been updated to add a note that says in part: "This is not real. I created these pics with AI (artificial intelligence) software. The children are not real, this never happened."

Contacted by AFP, the user referred to their posts and said the images were generated using Midjourney, a popular AI image tool.

The user went on to explain their process in additional posts on Facebook on May 9 here and here (archived links here and here).

"No, these are not pics actually taken. They were generated just in fun," the user wrote in one of the posts.

"There are no children that were harmed in these images because there were no children to begin with-it's all computer generated. Midjourney made these images for me," they added.

The post features a screenshot of an interaction with Midjourney's bot on Discord, in which the user included a prompt ordering the generation of an image of "children sitting on a carpet with a pentagram in the middle of a library".

Visual clues

AFP previously published a report on ways to distinguish genuine images from computer-generated ones, based on advice from specialists.

Experts pointed out common visual cues that can be used to identify AI-generated images -- such as inconsistencies, blurry sections or details in the hands -- as the technology remains limited in generating realistic limbs.

Some of these clues can be seen in the images shared in the misleading posts.

For example, some of the photos show more than five fingers on the children's hands, while other photos show inconsistent limbs and incomplete facial features.

Image

Scenes from two of the images shared in the misleading posts, showing inconsistent visual features

Unrelated to SatanCon

The Pumpkin Empress's May 9 Facebook post also denied the images were related to SatanCon. It reads: "This is not the satanic book club in Pennsylvania or wherever that was approved. This is not Satancon. I'm not a satanist."

SatanCon was an event convened in the US city of Boston from April 28 to 30, 2023 by The Satanic Temple, which defines itself as "the primary religious Satanic organization in the world with congregations internationally, and a number of high-profile public campaigns designed to preserve and advance secularism and individual liberties" (archived links here and here).

A spokesperson for The Satanic Temple told AFP by email on May 8, 2023 the images were created using AI and do not show the organisation's event.