Indonesians swindled by scams using President Prabowo deepfakes

"Who hasn't received aid from me? What are your needs right now?" Prabowo appears to ask viewers in the clip posted in November.

But while the Indonesian leader's mouth moves and his eyes blink, the words he utters are part of a fraudulent deepfake scam uncovered by police last month that has swindled Indonesians across 20 provinces (archived link).

Those ensnared by the message were asked to contact a WhatsApp number and hand over between 250,000 and one million rupiah ($15-$60) as an "administrative fee" to get aid that never materialised (archived link).

Image

Screenshot of a doctored video on Instagram

Since last year's Indonesian election, experts have warned of a tidal wave of deepfakes -- audio, images and video appearing to come from a known person but which are in fact the work of scammers using artificial intelligence tools.

And victims say the hoaxes are so sophisticated they leave others vulnerable to being conned too.

"People should be more careful. Don't be easily fooled by the lure of prizes," said Aryani, 56, who handed over 200,000 rupiah to fraudsters after seeing a deepfake video of a prominent Indonesian businessman.

"I need money, but instead I'm asked to send money. They even made video calls with me, as if I were talking directly to them."

During the Southeast Asian country's presidential campaign, deepfakes became a prominent tool to spread misinformation both harmful and helpful to candidates (archived link).

But now that technology has fallen into the hands of criminals looking to make cold, hard cash.

Wide circulation

AFP's fact-checkers found the account behind the Prabowo clip has posted dozens of similar videos appearing to show various high-profile figures, including Indonesian Vice President Gibran Rakabuming Raka.

Those videos also promote the bogus financial aid.

Police arrested a suspect who pocketed 65 million rupiah ($4,000) from the scam, Himawan Bayu Aji, director of the Indonesian National Cyber Crime Unit told reporters in February (archived link).

He said officers detained a second person involved in another scam that also used deepfake technology, without disclosing the amount raised.

AFP Fact Check's investigation found the spread of such videos had a much wider reach than the two accounts that police announced.

Deepfake videos of the president, a popular ex-general, were still circulating on social media after the arrests, including dozens on TikTok with the hashtag "Prabowo shares blessings".

At least 22 TikTok accounts were touting the same fraudulent scheme since Prabowo took office in October, AFP journalists found, with some appearing to take advantage of his recent rise to the presidency.

One account with more than 77,000 followers racked up 7.5 million views on a fabricated video of Prabowo apparently doling out financial assistance.

Another account with thousands of followers has shared 100 videos since January alone, the majority featuring deepfake videos of the president offering cash.

Image

Screenshot of a TikTok account sharing deepfake videos of Prabowo

TikTok said it had removed one of the deepfake scam videos and the associated account, adding it would continue to remove any that violated the platform's community guidelines, which prohibit misleading posts.

Facebook parent Meta did not respond to AFP's request for comment.

AFP, along with more than 100 other fact-checking organisations, is paid by TikTok and Meta to verify social media posts that potentially contain false information.

More accessible

Aribowo Sasmito, a co-founder of Indonesian fact-checking organisation Mafindo, said his team was finding new deepfake scams every week because of their prolific spread online.

"We have started to see deepfake videos since last year as AI tools have become more accessible and affordable," he told AFP.

Schemes using deepfakes of prominent figures appear to be on the rise.

Tech billionaire Elon Musk and Canada's Prime Minister Justin Trudeau are among a string of celebrities targeted in recent years by sophisticated impersonations to push cash scams.

Aribowo said his team of fact-checkers was working harder than ever to debunk these scams using the latest technology available to criminals.

"We noticed the quality of these videos has improved over time. It has become more difficult to distinguish between fake and real videos," he added.

"The challenge is the volume of these scams."