When code becomes abusive content, is it time for moderation?

GitHub took note of the social media furore and removed an egregious app’s code repository. It has, however, left coders wondering whether open-source platforms need rigorous moderation policies similar to the likes of Facebook.

A GitHub profile link acts as a resume for over 65 million developers around the world. If GitHub were a country, it would be the 22nd most populous in the world, ahead of the likes of France. More than 5.8 million developers use it to store and manage their code in India alone. One (or more) of them decided to use GitHub to host the code for an app called Sulli Deals.

Two months ago, several Muslim women known for being vocal on the internet, found their profiles on this app. “Sulli”, as one of them learnt for the first time through this incident, is a derogatory term for a Muslim woman. The app allowed users to express their intent to “find your ‘sulli’ of the day” by illicitly sourcing and uploading pictures and Twitter handles of multiple women from the Muslim community.

Women of any community are not alien to targeted abuse and harassment on social media platforms. GitHub, however, is unlike social media platforms. It is a community of engineers, not jobless trolls; a community that largely relies on the goodness of strangers to make their code better. “When one among us creates a product rooted in misogyny and Islamophobia, you feel insecure using the platform thinking that fellow-coders, who’ve had the privilege of education, could also be closet bigots,” says Shamsiya P., a software developer from Bengaluru.

Sulli Deals was an open-source project, which means the developers hosted the code on the platform for free, thus saving roughly $5-$10 a month on domain-hosting charges. Open-source projects at platforms like GitHub have other advantages, too. The domain name or the site’s URL has “github.io” suffixed to it, which increases its chances of appearing on top of search results. Keeping the code in the GitHub repository (github.com) and hosting it on github.io is also convenient for developers working on a low budget.

By the time it was discovered and taken down, the app had been live for a month, say a few coders who followed the development closely. Mint could not independently verify their claim. “The incident got me thinking,” says a Muslim woman software developer in Switzerland who prefers to stay anonymous. “Should the fundamentals of open-source philosophy, that promote creating, editing and redistributing software freely, add more clauses to its introductory slogan?”

As the perpetrators remain unknown and thus unpunished, despite two FIRs filed against them in Delhi and Uttar Pradesh, Muslim women coders in India and around the world say the incident has altered their relationship with the enabling tech platform that once felt like their home.

HATE IN THE OPEN

For too long, women have been treated as objects that can be commodified. The only difference is that so far, this practice was confined to the seedy underbelly of the online and offline market. “That someone thought they could put it out in the public domain—going to the extent of displaying its code to invite recommendations—scares me,” notes a Muslim woman coder from North America.

“GitHub has longstanding policies against content and conduct involving harassment, discrimination, and inciting violence,” a GitHub spokesperson told Mint. Coders we spoke to said GitHub doesn’t seem to widely communicate or actively implement them as such.

Is it time then for open-source platforms to introduce stringent moderation policies? “Without a doubt,” says Rashida K., a software developer from Europe, while highlighting that moderating code is far more challenging than moderating content, and perhaps ineffective in the absence of a legally binding framework.

To begin with, “you could create a basic code for an auction site on GitHub and store data (like pictures) elsewhere. It’ll be challenging for GitHub to track the abusive part of the code,” says Rashida who knew some of the women, whose profiles were on the now-defunct app, through Twitter. It’ll also require localised moderation to detect abusive terms confined to a region or community, she adds. “For instance, even I wasn’t aware of the term ‘Sulli’.”

It’s not impossible though. “These platforms can introduce community moderation,” says S. Shabeena, chief technology officer (CTO) at a social startup based in Vijayawada, Andhra Pradesh. “Even Wikipedia has community moderators assigned to monitor and filter everything uploaded on their platform.”

The solution will have to be a combination of manual and automated processes, adds Rashida. “Any open-source code repository platform should have the means to identify abusive content on its own as well as allow people to report it, not wait for 100 reports to initiate an investigation at their end.”

In July, Github announced an AI-powered code-completion tool, Copilot. Essentially, it is AI that learns from every line of code hosted on it and auto-completes code for developers (like the auto-word completion feature in smartphone keypads). “GitHub can come up with a similar program to flag inappropriate content,” says Fatima (surname withheld), a coder from Uttar Pradesh.

GitHub has been trying to make progressive changes to the platform in recent times, like replacing words such as “blacklist” and “whitelist” with “blocklist” and “allowlist”.

Fatima thinks building a code moderation tool is a long shot, “but I’m positive these companies are working on something like this.” Regardless, women coders “should start building this tech ourselves since we’re going to be the target of most of the inappropriate content anyway,” she says.