Crowdsourced Twitter Study Reveals Shocking Scale of Online Abuse Against Women

Amnesty International released a ground-breaking study into abuse against women on Twitter today, conducted with Element AI, a global artificial intelligence software product company. The findings showed that although the abuse cut across the political spectrum, women of color were much more likely to be impacted, and black women are disproportionately targeted.

More than 6,500 volunteers from 150 countries signed up to take part in Troll Patrol, a unique crowdsourcing project designed to process large-scale data about online abuse. Volunteers sorted through 228,000 tweets sent to 778 women politicians and journalists in the UK and the US in 2017.

Amnesty International and Element AI then used advanced data science and machine learning techniques to extrapolate data about the scale of abuse that women face on Twitter. Element AI calculated that 1.1 million abusive or problematic tweets were sent to the women in the study across the year - or one every 30 seconds on average.

"With the help of technical experts and thousands of volunteers, we have built the world's largest crowdsourced dataset about online abuse against women. Troll Patrol means we have the data to back up what women have long been telling us - that Twitter is a place where racism, misogyny and homophobia are allowed to flourish basically unchecked," said Milena Marin, Senior Advisor for Tactical Research at Amnesty International.

"Twitter's failure to crack down on this problem means it is contributing to the silencing of already marginalized voices."

Key findings

• Black women were disproportionately targeted, being 84% more likely than white women to be mentioned in abusive or problematic tweets. One in ten tweets mentioning black women was abusive or problematic, compared to one in fifteen for white women.
• 7.1% of tweets sent to the women in the study were problematic or abusive. This amounts to 1.1 million tweets mentioning 778 women across the year, or one every 30 seconds.
• Women of color, (black, Asian, Latinx and mixed-race women) were 34% more likely to be mentioned in abusive or problematic tweets than white women.
• Online abuse against women cuts across the political spectrum. Politicians and journalists faced similar levels of online abuse and we observed both liberals and conservatives alike, as well as left and right leaning media organizations, were affected.

Element AI used the data gathered in Troll Patrol to develop a machine learning model of an app which attempts to automatically detect abusive tweets. Element AI is making its model available to try out here for three weeks, to demonstrate the potential and current limitations of AI technology in this field.

"This study is part of a long-term partnership between Amnesty International and Element AI. Taking a sober approach to AI, we make long-term commitments, dedicating our technical experts and tools to enable social good actors to do what they do best," said Julien Cornebise, Director of Research, 'AI For Good' and Head of the London office of Element AI.

Politicians included in the sample came from across the US and UK political spectrums. The journalists included were from a diverse range of US and UK publications including The Daily Mail, The New York Times, Guardian, The Sun, GalDem, Pink News and Breitbart.

Amnesty International has repeatedly asked Twitter to publish data regarding the scale and nature of abuse on their platform, but so far the company has failed to do so. This hides the extent of the problem and makes it difficult to design effective solutions.

Troll Patrol volunteers collectively dedicated an incredible 2,500 hours analyzing tweets - the equivalent of someone working full-time for 1.5 years.

"By crowdsourcing research, we were able to build up vital evidence in a fraction of the time it would take one Amnesty researcher, without losing the human judgement which is so essential when looking at context around tweets," said Milena Marin.

"Troll Patrol isn't about policing Twitter or forcing it to remove content. We are asking it to be more transparent, and we hope that the findings from Troll Patrol will compel it to make that change. Crucially, Twitter must start being transparent about how exactly they are using machine learning to detect abuse, and publish technical information about the algorithms they rely on".

Background

Abusive content violates Twitter's own rules and includes tweets that promote violence against or threaten people based on their race, ethnicity, national origin, sexual orientation, gender, gender identity, religious affiliation, age, disability, or serious disease. Problematic content is defined as content that is hurtful or hostile, especially if repeated to an individual on multiple or cumulative occasions, but not as intense as an abusive tweet.

Amnesty International shared its findings with Twitter. In response, Twitter requested clarification on the definition of "problematic" "in accordance with the need to protect free expression and ensure policies are clearly and narrowly drafted."