Big tech blind spots leave young men vulnerable to online sexual extortion

Gaps in major platforms’ monitoring measures are allowing sexual extortion to thrive in Australia, the nation’s online safety regulator has found, as criminals repeatedly use the same methods to terrorise victims into handing over money.

These schemes typically target young men, tricking them into sharing intimate images of themselves before demanding payment in exchange for not making the material public. The motive is often quick financial gain, though the attacks often also result in new child sexual abuse material being circulated on the internet.

Despite the criminal tactics being known to investigators, and the number of victims growing, a report from the Office of the eSafety Commissioner released on Tuesday says platforms are not doing enough to mitigate the risks.

“We’re deeply concerned about the devastating impacts of sexual extortion, which not only targets vulnerable individuals but also has profound psychological and emotional consequences for victims and their families,” eSafety Commissioner Julie Inman Grant said in releasing the report.

“In several cases, we have provided these platforms with evidence of how their services are being colonised by criminals to devastating impact, with clear guidance on how to stem the abuse. Even when we’ve laid this out, we haven’t seen adequate responses, despite the technology being readily available.”

In the second half of last year, eSafety received 2206 complaints about sexual extortion, 85 per cent of them from men. Men aged 18 to 24 were the single largest group. Previous research has shown adolescents are also targeted, with more than one in 10 reporting they had faced sexual extortion. Of those, more than half had experienced it before the age of 16.

The report, which is the third in a series of four analysing data that companies including Apple, Meta, Google and Microsoft are obliged to provide in relation to Australia’s Basic Online Safety Expectations, points to a number of major gaps where protections could be improved.

Big technology companies have previously argued they are dedicated to stamping out sexual exploitation on their platforms. Many let parents place additional safeguards on accounts belonging to young people, offer help centres to users and employ safety teams to flag and remove illegal material. They also argue that user privacy is important, and have resisted mass surveillance on some of their sites.

The commission, though, says reporting tools and warnings offered by tech companies are frequently clunky and ineffective. Some apps lack dedicated reporting categories for sexual extortion, or actively deter victims from seeking help with complex web forms. Many can detect suspect behaviour to warn potential victims, but those measures aren’t necessarily effective. On Snapchat, more than 99 per cent of adolescents ignored warnings to view messages from high-risk accounts.

Offenders also often use a “kill chain”, initiating contact on public apps like Tinder or Instagram, before migrating to encrypted channels like WhatsApp, which protect user privacy but make it more difficult for criminals to be exposed. Extortionists also rely on highly repetitive scripts, which eSafety said should be detectable with language analysis technology.

The regulator also said that, except for Microsoft Teams, no major online platform proactively monitors or disrupts exploitation occurring via video calls.

“Offenders are continuing to exploit gaps in platform design, weak detection systems and inconsistent safeguards to move seamlessly between services and escalate harm against children. This report shows that platforms could and should be doing a lot more to prevent these harms, and there are simple steps they can take today to protect users,” Inman Grant said.

“These are some of the most innovative companies on the planet, with some of the best minds. We would like to see some of this innovation going into the development of new technologies to tackle the worst of the worst online content.”

Suggestions for improvement listed in the report include text analysis that can recognise known extortion scripts, monitoring to detect backend giveaways of suspicious behaviour like sudden geographical shifts, and frictionless one-click in-app reporting. For encrypted platforms, eSafety suggested the development of on-device abuse detection that could flag conversations to an end user without affecting privacy.

Victims of sexual extortion are encouraged to stop contact immediately, collect evidence, report it and not pay. If intimate content is shared online, it can be reported to eSafety, which has powers to remove it. Children under 18 can report to the Australian Centre to Counter Child Exploitation.

Kids Helpline: 1800 55 1800

Headspace: 1800 650 890

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.