The Deepfake Porn Crisis Affecting Korean Schools

Last Saturday, a Telegram message from an anonymous sender appeared on Heejin’s phone. “Your photos and personal information have been leaked. Let’s discuss it.”

When the college student entered the chat room to read the message, she received a photo of herself taken a few years ago, when she was still in school. This was followed by a second image with the same photo, only this one was sexually explicit and fake.

Terrified, Heejin, not her real name, did not respond, but the images kept coming. In all of them, her face had been attached to a body in the act of sex, using sophisticated technology. deepfake.

The deepfakeswhich combine a real person’s face with a sexually explicit fake body, are increasingly being generated using artificial intelligence.

“I was petrified and felt so alone,” Heejin told the BBC.

Two days earlier, South Korean journalist Ko Narin had published what would become the biggest scoop of her career. Police had recently been investigating fake pornography rings at two of the country’s top universities, and Ko was convinced there had to be more.

He began searching social media and discovered dozens of chat groups on the messaging app Telegram, where users shared photos of women they knew and used artificial intelligence software to turn them into fake pornographic images in seconds.

“Every minute people were uploading photos of girls they knew and asking to be turned into deepfakes“Ko said.

Ko discovered that these groups weren’t just aimed at female college students. There were rooms dedicated to certain high schools and even colleges. If a lot of content was created with images of a particular student, she could even be assigned a room of her own. These virtual spaces, called “humiliation rooms” or “friends of friends” rooms, are usually subject to strict access conditions.

Ko’s report in the Hankyoreh newspaper has shocked South Korea. On Monday, police announced they were considering opening an investigation into Telegram, following the example of French authorities, who charged Telegram’s Russian founder with crimes related to the app. The government has promised to impose harsher penalties on those involved and the president has called for better education for young people.

In a statement provided to the BBC, Telegram said it “actively combats harmful content on its platform, including illegal pornography.”

The BBC has seen descriptions of several of these chats. One asks members to post more than four photos of someone, along with their name, age and the area in which they live.

“I was shocked at how systematic and organized the process was,” Ko said. “The most horrifying thing I discovered was a group for underage students at a school that had more than 2,000 members.”

In the days following the publication of Ko’s article, women’s rights activists also began scouring Telegram and following up on leads.

By the end of that week, More than 500 schools and universities had been identified as targetsThe actual number of those affected has not yet been determined, but many are believed to be under 16, the age of consent in South Korea. Many of the suspected perpetrators are teenagers.

Heejin said that knowing the magnitude of the crisis had compounded her anxiety, as she was now worried about how many people might have seen her deepfakesAt first she blamed herself. “I couldn’t stop thinking if this had happened because I had posted my photos on social media, should I have been more careful?”

Since then, dozens of women and teenagers across the country have deleted their photos from social media or deactivated their accounts, fearing they could be next to be exploited.

“We are frustrated and angry that we have to censor our behavior and our use of social media when we have done nothing wrong,” said Ah-eun, a university student whose fellow students have been targeted.

Ah-eun explained that one victim at her university was told by police not to bother pursuing her case because it would be too difficult to catch the perpetrator. Incidentally, “it wasn’t really a crime” because “the photos were fake.”

At the centre of this scandal is the messaging app Telegram. The app is known for having a “light touch” moderation stance and has for years been accused of not doing enough to control content and, in particular, groups.

This has made it a prime space for criminal conduct to flourish.

Last week, politicians and police responded forcefully, promising to investigate these crimes and bring their perpetrators to justice.

On Monday, Seoul’s National Police Agency announced it would investigate Telegram for its role in distributing fake pornographic images of children.

The app’s founder, Pavel Durov, was charged in France last week with being complicit in several crimes related to the app, including allowing the sharing of child pornography.

But Women’s rights activists accuse South Korean authorities of allowing sexual abuse on Telegram simmer unchecked for too long, because Korea has faced this crisis before.

In 2019, it came to light that a sex network was using Telegram to coerce women and children into creating and sharing sexually explicit images of themselves.

Police then asked Telegram for help in their investigation, but the app ignored all seven of their requests. Although the ringleader was sentenced to more than 40 years in prison, no action was taken against the platform due to fears related to censorship.

“They convicted the main perpetrators but they ignored the situation, and I think this has made the situation worse,” Ko said.

Park Jihyun, who exposed the Nth dorm sex ring in 2019 when she was a journalism student, has become a political advocate for victims of digital sex crimes. She says that since the deepfake scandal broke, students and parents have called her several times a day crying.

“They’ve seen their school on the list shared on social media and they’re terrified.”

Park has led calls for the government to regulate or even ban the app in South Korea. “If these tech companies do not cooperate with law enforcement, The State must regulate them to protect its citizens“, he said.

Before this latest crisis erupted, South Korea’s Online Sexual Abuse Victims Advocacy Center was already noticing a sharp rise in the number of underage victims of pornography. deepfake.

In 2023, they advised 86 teenage victims. In the first eight months of this year, the number has risen to 238. In the last week alone, another 64 teenage victims have come forward.

One of the center’s managers, Park Seonghye, said that Last week his staff had been inundated with calls and were working tirelessly.“For us, it was a large-scale emergency, like a war situation,” he said.

“With the latest technology deepfake“There are many more images now than before, and we are concerned that they will increase.”

In addition to advising victims, the center tracks down harmful content and works with online platforms to remove it. Park said that in some cases Telegram has removed content at the request of victims. “So it’s not impossible,” she said.

In a statement, Telegram told the BBC that its moderators “proactively monitor public parts of the app, use artificial intelligence tools and accept user reports to remove millions of pieces of content a day that violate Telegram’s terms of service.”

While women’s rights organisations accept that new artificial intelligence technology facilitates the exploitation of victims, they argue that this is the latest form of misogyny to occur in South Korea.

First, women suffered waves of verbal abuse online. Then came the epidemic of spy cameras, where they were secretly filmed in public toilets and changing rooms.

“The root of all this is structural sexism, and the solution is gender equality“, read a statement signed by 84 women’s groups.

This is a direct criticism of the country’s President Yoon Suk Yeol, who has denied the existence of structural sexism, cut funding to victim support groups and is abolishing the government’s Ministry of Gender Equality.

Lee Myung-hwa, who treats young sex offenders, agreed that although the outbreak of deepfakes It may seem sudden, but it has been lurking beneath the surface for a long time.

“For teenagers, the deepfakes “It has become part of their culture, they see it as a game or a joke,” said the consultant, who runs the Aha Seoul Youth Cultural Center.

Lee said it was essential to educate young people. She said studies show that when offenders are told what they have done wrong, they are more aware of what is considered sexual abuse and do not reoffend.

Meantime, The government has announced that it will increase penalties for those who create and share fake images. He will also punish those who see them.

This is due to criticism for not punishing the perpetrators. One of the problems is that the offenders are mostly teenagers, who are usually tried in juvenile courts, where they receive lighter sentences.

Since the chat rooms were discovered, many have been closed, but new ones are almost certain to take their place. A humiliation room has already been set up for journalists covering this story. Ko, who broke the story, says it has kept him up at night. “I keep checking the room to see if my photo has been posted,” he explains.

This anxiety has spread to almost all teenage girls and young women in South Korea. Ah-eun, a university student, said it had made her wary of male acquaintances.

“Now I can’t be sure that these crimes aren’t being committed behind my back, without my knowledge,” she says. “I’ve become hypervigilant in all my interactions with people, which can’t be good.”

Additional reporting by Hosu Lee and Suhnwook Lee

And remember that you can receive notifications in our app. Download the latest version and activate them.