Facebook has launched an effort targeting users working together on the platform to promote real-world violence or conspiracy theories, beginning by taking down a German network spreading Covid misinformation.
The new tool announced Thursday is meant to detect organized, malicious efforts that are a threat but fall short of the social media giant’s existing rules against hate groups, said Facebook’s head of security policy Nathaniel Gleicher.
Facebook has been under relentless pressure to guard against being a platform where misinformation and hate can spread, while at the same time remain a forum for people to speak freely. It has struggled to respond.
Under the new effort, users who work together to “amplify their group’s harmful behavior” and repeatedly violate the platform rules can have their accounts shut down.
Facebook is looking for groups of users that do things like “brigading,” or ganging up on other accounts, to flood them with comments or complaints.
“We recognize this challenge is complex,” Facebook threat disruption director David Agranovich told a press briefing.
“We need to be careful and deliberate… to distinguish between people who organically come together to organize for social change, and the types of adversarial networks that can cause social harm,” he added.
A series of recent Wall Street Journal articles has cast a harsh light on the company for failing to protect teenage users of its photo app Instagram, but also for shielding VIPs from some of the network’s own restrictions.
Under the new effort, Facebook has removed fewer than 150 accounts, pages, or groups operated by people associated with the Querdenken movement, which opposes anti-Covid measures like mask-wearing and lockdowns.
People behind the accounts, some of which were on Instagram, boosted content that portrayed violence as the way to overturn German government efforts against the virus, according to Facebook.
The social network cited public reports that the group took part in violence against journalists, police and medical practitioners in Germany.
The new enforcement tool will take aim at groups with histories of violating Facebook’s rules and trying to dodge accountability.
Gleicher said the network has been developing this new tool since before the start of this year, as harmful social media campaigns increasingly enlisted real users to spread posts.
He noted bad actors “deliberately blur the lines” between real people expressing their ideas and deliberate manipulation, in order to be harder to catch.
