Automated Moderation Is Here to Stay—Accountability Must Keep Pace

EFF update
1 hour 34 minutes ago

This post is part 2 in a series about automated content moderation. Read the first post here.

When whistleblower Frances Haugen leaked a set of documents from Meta in 2020, among the revelations was a jarring statistic: The company’s algorithms designed to detect terrorist content incorrectly deleted nonviolent Arabic-language content 77 percent of the time, while failing to detect hate speech under the company’s own policies in many instances. Meta’s own transparency report released later that year demonstrated similar findings. Five years later, researchers in the region report that overzealous moderation remains a problem, while paths to remedy have all but collapsed.

Where these systems are faltering in Arabic, they’re positively failing in less-resourced languages. As a 2025 report from the Center for Democracy and Technology found, labeled datasets in certain languages and dialects such as Maghrebi Arabic and Kiswahili contain inconsistencies, bias, and inaccuracies due to the limited hiring of annotators who actually speak the languages as well as shifts in the languages themselves. An investigation into ChatGPT’s outputs in several low-resource languages demonstrates the depth of problem.

But language disparities are just one of several concerns as automated moderation becomes more widespread. From the systemic suppression of content from Palestine to the repeated misclassification of LGBTQ+ content as adult or explicit material, these varied examples demonstrate the risks of overreliance on automated moderation—and the need for stronger safeguards.

Transparency, Cultural Competence, Appeals

As we discussed in Part 1 of this series, automated systems can process content at a scale that humans never could, potentially enabling better moderation at scale and alleviating the psychological load on ill-paid moderators whose jobs require them to view incredibly disturbing content. But automated systems also reproduce existing biases, struggle to understand context, and often make mistakes that disproportionately affect journalists, activists, artists, and other vulnerable and marginalized communities.

As Rachel Griffin wrote in 2023, “Perfectly accurate moderation is not only technically out of reach but intrinsically impossible.” Despite those intrinsic flaws, there is a great deal companies, policymakers, and civil society can do to help ensure that highly-automated systems operate in ways that respect human rights, minimize predictable harms, and provide meaningful accountability when they fail. If companies are going to continue relying on automation to moderate users’ speech—and there is little reason to believe they won’t—then accountability must evolve alongside these technologies.

That evolution can start with committing to the Santa Clara Principles 2.0. These principles, first outlined in 2020 and re-launched in 2021 after substantial international input, reflect the needs and expectations of the global community and specifically address automation. The first Foundational Principle states:

Companies should ensure that human rights and due process considerations are integrated at all stages of the content moderation process, and should publish information outlining how this integration is made. Companies should only use automated processes to identify or remove content or suspend accounts, whether supplemented by human review or not, when there is sufficiently high confidence in the quality and accuracy of those processes. Companies should also provide users with clear and accessible methods of obtaining support in the event of content and account action. 

Drawing on the Santa Clara Principles 2.0, international human rights standards, and years of research documenting the shortcomings of automated moderation, we propose eight recommendations for policymakers thinking about regulation and companies deploying AI-assisted content moderation systems.

  1. Automated technologies should help, not replace, human moderators. For example, automated systems can help flag and prioritize content for review, while humans can interpret context, handle sensitive cases, and refine system performance.
  2. Companies must be transparent about when and how automation is used in content decisions.
  3. Companies must regularly audit their automated systems for bias, with particular attention to low-resource languages, vulnerable and marginalized communities, and conflict zones.
  4. Users must have the ability to appeal, and to provide context when they believe human or automated moderation decisions have wrongfully removed their content. Appeals should be promptly evaluated and decided by human moderators.
  5. Companies should regularly assess the human rights impact of their moderation decisions, and issue public statements of the results
  6. If they rely on third-party vendors, companies should carefully (and regularly) audit those vendors for compliance with these same principles
  7. Lawmakers should avoid promoting and passing legislation that effectively or explicitly mandates automated moderation systems
  8. Policymakers should also refrain from attempting to dictate platforms technical and design choices to favor or disfavor particular expression.

These recommendations understand that automated content moderation isn’t just a technical problem for clever engineers and product teams to solve. Because content moderation shapes public discourse and fundamental rights, its design and oversight must respond to the concerns of policymakers, civil society, independent researchers, and the communities most affected by these systems.

This is the second post in a 2-part series on automated content moderation. Read the first post here.

Jillian C. York

"We Want Texans to Know Their Rights": Q&A with Mayday Health on the Impact of Surveillance on Abortion Care

EFF update
17 hours 40 minutes ago

Last May, EFF reported that a sheriff’s office in Texas searched data from more than 83,000 automated license plate reader (ALPR) cameras to track down a woman suspected of self-managing an abortion. ALPRs are promoted as tools for keeping communities safe by finding missing persons and locating stolen vehicles, but this case showed how ALPRS can be weaponized to investigate people’s private healthcare decisions. And these aren’t the only tools in the surveillance arsenal: others include location tracking tools like Locate X, which can show a person’s visit to an abortion clinic, or search histories which might be used as evidence of a person’s interest in obtaining abortion pills. Taken together, these tools create a dangerous surveillance pipeline that threatens everyone’s health privacy. 

Too often, though, the public is unaware of the threat, and one nonprofit is working to change that. Following EFF and 404 Media’s report on Texas’s use of Flock cameras, eye-catching billboards popped up in Houston, warning drivers that if they’re pregnant, the state of Texas could be tracking them.  


Photo provided by Mayday Health

These billboards came from Mayday Health, a nonprofit dedicated to sharing information about abortion pills, birth control, and gender-affirming care. We spoke with Leo Raisner, Executive Director of Mayday Health, about the billboards to learn more about the campaign and organization and to discuss how surveillance affects reproductive freedom. 

***

THOMAS: Why did Mayday Health start this campaign in Texas?  

RAISNER: Well, we read the incredible reporting coming from EFF about Texas's surveillance. We want Texans to know their rights, to know their options, and to know that there are organizations and people who have their back. So we decided to put up a few billboards around the Houston area to remind people that they still have options.  

Digital advertising in the space, as I know you're well aware of, faces enormous platform restrictions from Meta and Google, whereas billboards reach people in the physical world without algorithmic gatekeeping and without requiring anyone to search for information. So at the very least, if a driver's passing by the billboard, we’re spreading information that they should be careful that they might be surveilled, and also there are different options. There's a website where they can come learn more about those options. 

THOMAS: And how have the billboards been received so far? Have you heard anything from folks in the Houston area yet?    

RAISNER: Yeah, we've heard some messages of support on social media DMs. We're just thrilled about how many drivers these messages are going to reach. They'll be up for 4 weeks, and are expected to hit over 1,000,000 drivers during that 4-week campaign period.    

THOMAS: Are there other ways that Mayday Health has seen surveillance systems impact people seeking healthcare?  

RAISNER: You know, we go all over the country and talk to folks who are seeking reproductive healthcare options in states where clinics are banned, and we direct folks to our website where they can learn more about abortion pills. We make privacy very central to how we operate. Privacy is not just an afterthought for us. When people arrive at our website, we direct them to the Digital Defense Fund, which offers people privacy and security resources as they're navigating reproductive healthcare in states where they might be being surveilled. We don't collect cookies, we don't collect identifying information from visitors to our site. We want people to know their options, and we don't have any interest in knowing who they are.  

THOMAS: Why do you think the work of the digital rights movement is so important to the work of the reproductive health rights and justice movement?  

RAISNER: I mean, those two movements are inextricably linked. The anti-abortion movement is using every tool in their toolbox to prevent people from getting the healthcare access they need, whether that's surveilling people online or closing down brick-and-mortar clinics, but we encourage people to visit Mayday Health and learn that they still have options no matter where they live.  

THOMAS: Is there anything else that you would like the readers of our blog to know about Mayday Health?    

RAISNER: I'd love for people to know that abortion pills are FDA approved. They're safe, they're effective, and they're available through the mail. 

*** 

EFF has said it time and time again – surveillance and reproductive freedom cannot coexist. Whether the tracking occurs over the internet or through license plate reader systems with over 83,000 cameras, it is an invasion of privacy. Protecting our digital privacy is more critical now than ever. Help EFF fight back against this digital dragnet and protect reproductive freedom for all by making a donation. 

Kenyatta Thomas

The House Passed The KIDS Act—The Senate Should Reject It 

EFF update
17 hours 55 minutes ago

Last week, the House voted on the KIDS Act, a disjointed package of legislation that seeks to control Americans’ web browsing and private messaging. The package combines a revised version of the Kids Online Safety Act (KOSA), with several other internet bills, study bills, reporting requirements, and new regulations. Different parts of the bill pressure online services to impose different age-gating schemes, using different standards. EFF opposed this bill, along with many of our members and supporters.

Take action

Tell Congress: no internet age-gates

The bill passed the House, 267-117. It now heads to the Senate, where its fate remains uncertain. But this fight is not over. Even if you took our earlier action to contact the House, we need you to reach out to your Senators today. 

The KIDS Act Will Lead to Mandatory Age Checks 

Many of the bills in the KIDS Act share the same premise: that children and teenagers should have different experiences online than adults. In practice, that requires websites and apps to determine who is under 18—and who isn’t. That’s where the problems with the KIDS Act start. 

EFF certainly supports giving all users better privacy and safety tools online. But those protections should not, and do not need to, come at the expense of privacy or free expression. Unfortunately, that’s exactly the tradeoff the KIDS Act makes.

There is no way to determine a user’s age online that is both privacy protective and accurate. Some age verification processes may rely on collecting government-issued ID, while others may use biometric scans. Others will use algorithms to guess a user’s age based on facial images or online behavior. But no matter the method, every system demands users hand over sensitive personal information that links their offline identity to their online activity. And then, once that valuable data is collected, it can be leaked, hacked, or misused. In fact, we’ve already seen several breaches of age verification providers.

The Bill Still Regulates Online Speech

The revised KOSA language within the KIDS Act still pressures companies to police lawful speech online. Platforms must “establish, implement, maintain, and enforce” policies that address content like gambling or the use of alcohol or cannabis. This encourages platforms to broadly restrict speech on these topics, which could include a teen seeking advice on a parent’s gambling problem or searching for substance abuse recovery resources. When platforms are required to create and enforce content moderation policies that regulators can sue them over, they will often err on the side of deleting speech. 

Protect Privacy For Everyone

There is a better way to protect young people online. Instead of encouraging a complicated system of age checks, more monitoring, and more restrictions on access to information, Congress could finally pass a strong, comprehensive privacy law that benefits all users. A great place to start would be to ban behavioral advertising that tracks us across the web—again, for users of all ages. 

We urge the Senate to oppose the KIDS Act and instead focus on a strong, bipartisan privacy package for all users. 

Take action

Tell the senate to reject the kids act

India McKinney