Connect with us

Hi, what are you looking for?

Social Media

New tool balances accuracy with fairness in social media

An algorithm may be accurate at detecting toxic speech overall, but it may not detect it equally well across all groups of people and all social contexts.

Australian legislation could force social media firms to take steps to prevent those under 16 years of age from accessing platforms such as X, TikTok, Facebook and Instagram
Australian legislation could force social media firms to take steps to prevent those under 16 years of age from accessing platforms such as X, TikTok, Facebook and Instagram - Copyright GETTY IMAGES NORTH AMERICA/AFP/File Michael M. Santiago
Australian legislation could force social media firms to take steps to prevent those under 16 years of age from accessing platforms such as X, TikTok, Facebook and Instagram - Copyright GETTY IMAGES NORTH AMERICA/AFP/File Michael M. Santiago

Earlier this year, Facebook rolled back rules against some hate speech and abuse. Along with changes at X (formerly Twitter) that followed its purchase by Elon Musk, the shifts make it harder for social media users to avoid encountering toxic speech.

That does not mean all social networks and other online spaces have given up on the massive challenge of moderating content to protect users. One novel approach relies on artificial intelligence. AI screening tools can analyze content on large scales while sparing human screeners the trauma of constant exposure to toxic speech.

Yet AI content moderation faces a challenge, according to Maria De-Arteaga, assistant professor of information, risk, and operations management at Texas McCombs. This is with: being fair as well as being accurate.

An algorithm may be accurate at detecting toxic speech overall, but it may not detect it equally well across all groups of people and all social contexts.

“If I just look at overall performance, I may say, oh, this model is performing really well, even though it may always be giving me the wrong answer for a small group,” De-Arteaga explains. For example, it might better detect speech that’s offensive to one ethnic group than to another.

In new research, De-Arteaga and her co-authors show it’s possible to achieve high levels of both accuracy and fairness. What’s more, they devise an algorithm that helps stakeholders balance both, finding desirable combinations of accuracy and fairness for their particular situations. De-Arteaga worked with datasets of social media posts already rated “toxic” and “nontoxic” or safe by previous researchers. The sets totaled 114,000 posts.

The researchers used a fairness measurement called Group Accuracy Parity (GAP), along with formulas that helped train a machine learning model to balance fairness with accuracy. Applying their approach through AI to analyze the datasets:

  • It performed up to 1.5% better than the next-best approaches for treating all groups fairly.
  • It performed the best at maximizing both fairness and accuracy at the same time.

But GAP is not a one-size-fits-all solution for fairness, De-Arteaga notes. Different measures of fairness may be relevant for different stakeholders. The kinds of data needed to train the systems depends partly on the specific groups and contexts for which they’re being applied.

For example, different groups may have different opinions on what speech is toxic. In addition, standards on toxic speech can evolve over time.

Getting such nuances wrong could wrongly remove someone from a social space by mislabeling nontoxic speech as toxic. At the other extreme, missteps could expose more people to hateful speech.

The challenge is compounded for platforms like Facebook and X, which have global presences and serve wide spectrums of users.

“How do you incorporate fairness considerations in the design of the data and the algorithm in a way that is not just centered on what is relevant in the U.S.?” De-Arteaga says.

For that reason, the algorithms may require continual updating, and designers may need to adapt them to the circumstances and kinds of content they’re moderating, she says. To facilitate that, the researchers have made GAP’s code publicly available.

High levels of both fairness and accuracy are achievable, De-Arteaga says, if designers pay attention to both technical and cultural contexts.

“You need to care, and you need to have knowledge that is interdisciplinary,” she says. “You really need to take those considerations into account.”

The articleFinding Pareto Trade-Offs in Fair and Accurate Detection of Toxic Speech” is published in Information Research.

Avatar photo
Written By

Dr. Tim Sandle is Digital Journal's Editor-at-Large for science news. Tim specializes in science, technology, environmental, business, and health journalism. He is additionally a practising microbiologist; and an author. He is also interested in history, politics and current affairs.

You may also like:

Tech & Science

Masking, distancing and quarantines keep chimpanzees safe from human disease, a new study shows.

Tech & Science

Nipah virus disease in humans is associated with significant morbidity and a high mortality rate and consequent public health impact.

Entertainment

Filmmaker Johnny Martin chatted about directing the new action movie "Off the Grid," which was released on June 27th via Lionsgate.

World

While the heftiest of Donald Trump's tariffs on Vietnam have been avoided, there are worries about the effect of the deal with regard to...