Tumblr and the New York’s Commission on Human Rights (CCHR) have settled allegations of discrimination concerning the company’s recent adult material ban. The city officials claim disproportionately affected LGBTQ users. The agreement calls for Tumblr to review the procedure for submitting appeals to its users, and also to educate its moderators on diversity and inclusive concerns, and to examine thousands of cases from the past and hire an expert investigate any potential bias within its moderator algorithms.
The settlement, which does not require a formal legal action and was signed in December, is one of the very first occasions regulators reached an agreement for the change of the policies of moderation on social networks that are based on bias in algorithm. This settlement resolves an inquiry that had been conducted by the CCHR started in December of 2018 just following the time that Tumblr was banned from posting explicit sexual material and nakedness — and enforced the rules using the use of a hilariously inaccurate automated system for taking down content.
“If someone is doing business in New York City, we have the authority to investigate”
In an interview in an interview with the Verge, CCHR press secretary Alicia McCauley says the agency was interested following reports suggesting that the ban could cause had an significant impact on the Tumblr’s LGBTQ members. McCauley states that the city’s Human Rights Law provides broad security against discrimination on the basis of aspects like gender identity as well as sexual orientation. “If someone is doing business in New York City, we have the authority to investigate if it’s negatively affecting people,” McCauley said.
The agreement gives Tumblr an extra 180 days to find an expert in the field of gender identity and sexual orientation (SOGI) concerns as well as add the necessary training for moderators. They must also employ an expert with expertise in this field and also have an expert in image classification that will analyze Tumblr’s moderating algorithm to determine if it is more likely to block LGBTQ material. For the purposes of the general review Tumblr will examine 3,000 older instances where users had a successful appeal against a takedown and look for signs which could suggest bias.
The acquisition seems to have been arranged in large part due to WordPress.com owner Automattic which bought Tumblr through Verizon in 2019. The company appeared to have worked closely with CCHR. “I think that was a turning point in the investigation,” claims CCHR lawyer Alberto Rodriguez. Automattic has redesigned the initial method to include an extra human-controlled oversight prior to the settlement. As part of its new ownership, Tumblr has tried to find a way of reuniting the LGBTQ users who left in the context of the large community dispersal.
Rodriguez believes that the Tumblr settlement may be an initial step toward the larger regulatory effort across the country. “I think it’s inevitable that social media companies are going to come under more government regulation and that more of these enforcement actions are going to come about,” Rodriguez says.
Cases involving bias in social media are not often successful in court.
Social media’s bias allegations against platforms have not always been successful in the courts, but today’s settlement appears to be supported by the desire of Automattic to revamp the Tumblr moderation system and restore faith in its LGBTQ community. (Automattic is a tiny company, with far fewer legal resources as “Big Tech” giants.) The CCHR did not offer specifics on the evidence supporting its assertions of discrimination. It’s hard to judge the particulars of the situation. However, larger platforms such as YouTube and Instagram are also facing accusations of discriminatory moderation, but without any regulation, and YouTube specifically has won two suits brought by LGBTQ as well as Black video makers who claimed that algorithmic discrimination.
Rodriguez states that, unlike these instances, the CCHR’s municipal-level regulations don’t need a particular purpose to make discrimination a reality. The courts also have granted social media platforms the ability to regulate material in accordance with Section 230 of the First Amendment and Section 230 of the Communications Decency Act, and any CCHR suit would need to be able to stand the review. “Section 230 applies equally to federal, state, and municipal laws and enforcement,” states Jeff Kosseff, author of extensive Section 230 history The Twenty-Six Words that Created the Internet.
The bigger problem of algorithms that are based on gender and race discrimination is increased in importance for authorities, specifically when it could influence people’s job and housing choices. In spite of no legal concerns some firms like Twitter have been forced to review their moderation processes after public pressure — at times uncovering troubling information in the process.