NYC Lets AI Gamble with Child Welfare

4 days 16 hours ago

The Markup revealed in its reporting last month that New York City’s Administration for Children’s Services (ACS) has been quietly deploying an algorithmic tool to categorize families as “high risk". Using a grab-bag of factors like neighborhood and mother’s age, this AI tool can put families under intensified scrutiny without proper justification and oversight.

ACS knocking on your door is a nightmare for any parent, with the risk that any mistakes can break up your family and have your children sent to the foster care system. Putting a family under such scrutiny shouldn’t be taken lightly and shouldn’t be a testing ground for  automated decision-making by the government.

 This “AI” tool, developed internally by ACS’s Office of Research Analytics, scores families for “risk” using 279 variables and subjects those deemed highest-risk to intensified scrutiny. The lack of transparency, accountability, or due process protections demonstrates that ACS has learned nothing from the failures of similar products in the realm of child services.

The algorithm operates in complete secrecy and the harms from this opaque “AI theater” are not theoretical. The 279 variables are derived only from cases back in 2013 and 2014 where children were seriously harmed. However, it is unclear how many cases were analyzed, what, if any, kind of auditing and testing was conducted, and whether including of data from other years would have altered the scoring.

What we do know is disturbing: Black families in NYC face ACS investigations at seven times the rate of white families and ACS staff has admitted that the agency is more punitive towards Black families, with parents and advocates calling its practices “predatory.” It is likely that the algorithm effectively automates and amplifies this discrimination.

Despite the disturbing lack of transparency and accountability, ACS’s usage of this system has subjected families that this system ranks as “highest risk” to additional scrutiny, including possible home visits, calls to teachers and family, or consultations with outside experts. But those families, their attorneys, and even caseworkers don't know when and why the system flags a case, making it difficult to challenge the circumstances or process that leads to this intensified scrutiny.

This is not the only incidence in which usage of AI tools in the child services system has encountered issues with systemic biases. Back in 2022, the Associated Press reported that Carnegie Mellon researchers found that from August 2016 to May 2018, Allegheny County in Pennsylvania used an algorithmic tool that flagged 32.5% of Black children for “mandatory” investigation compared to just 20.8% of white, all while social workers disagreed with the algorithm's risk scores about one-third of the time.

The Allegheny system operates with the same toxic combination of secrecy and bias now plaguing NYC. Families and their attorneys can never know their algorithmic scores, making it impossible to challenge decisions that could destroy their lives. When a judge asked to see a family’s score in court, the county resisted, claiming it didn't want to influence legal proceedings with algorithmic numbers, which suggests that the scores are too unreliable for judicial scrutiny yet acceptable for targeting families.

Elsewhere these biased systems were successfully challenged. The developers of the Allegheny tool had already had their product rejected in New Zealand, where researchers correctly identified that the tool would likely result in more Māori families being tagged for investigation. Meanwhile, California spent $195,273 developing a similar tool before abandoning it in 2019 due in part to concerns about racial equity.

Governmental deployment of automated and algorithmic decision making not only perpetuates social inequalities, but removes mechanisms for accountability when agencies make mistakes. The state should not be using these tools for rights-determining decisions and any other uses must be subject to vigorous scrutiny and independent auditing to ensure the public’s trust in the government’s actions.

Hannah Zhao

Criminalizing Masks at Protests is Wrong

4 days 17 hours ago

There has been a crescendo of states attempting to criminalize the wearing of face coverings while attending protests. Now the President has demanded, in the context of ongoing protests in Los Angeles: “ARREST THE PEOPLE IN FACE MASKS, NOW!”

But the truth is: whether you are afraid of catching an airborne illness from your fellow protestors, or you are concerned about reprisals from police or others for expressing your political opinions in public, you should have the right to wear a mask. Attempts to criminalize masks at protests fly in the face of a right to privacy.

Anonymity is a fundamental human right.

In terms of public health, wearing a mask while in a crowd can be a valuable tool to prevent the spread of communicable illnesses. This can be essential for people with compromised immune systems who still want to exercise their First Amendment-protected right to protest.

Moreover, wearing a mask is a perfectly legitimate surveillance self-defense practice during a protest. There has been a massive proliferation of surveillance camera networks, face recognition technology, and databases of personal information. There also is a long law enforcement’s history of harassing and surveilling people for publicly criticizing or opposing law enforcement practices and other government policies. What’s more, non-governmental actors may try to identify protesters in order to retaliate against them, for example, by limiting their employment opportunities.

All of this may chill our willingness to speak publicly or attend a protest in a cause we believe in. Many people would be less willing to attend a rally or march if they know that a drone or helicopter, equipped with a camera, will take repeated passes over the crowd, and police later will use face recognition to scan everyone’s faces and create a list of protest attendees. This would make many people rightfully concerned about surveillance and harassment from law enforcement.

Anonymity is a fundamental human right. EFF has long advocated for anonymity online. We’ve also supported low-tech methods to protect our anonymity from high-tech snooping in public places; for example, we’ve supported legislation to allow car owners to use license plate covers when their cars are parked to reduce their exposure to ALPRs.

A word of caution. No surveillance self-defense technique is perfect. Technology companies are trying to develop ways to use face recognition technology to identify people wearing masks. But if somebody wants to hide their face to try to avoid government scrutiny, the government should not punish them.

While members of the public have a right to wear a mask when they protest, law enforcement officials should not wear a mask when they arrest protesters and others. An elementary principle of police accountability is to require uniformed officers to identify themselves to the public; this discourages officer misconduct, and facilitates accountability if an officer violates the law. This is one reason EFF has long supported the First Amendment right to record on-duty police, including ICE officers.

For these reasons, EFF believes it is wrong for state legislatures, and now federal law enforcement, to try to criminalize or punish mask wearing at protests. It is especially wrong, in moments like the present, where government it taking extreme measures to crack down on the civil liberties of protesters. 

Matthew Guariglia

Privacy Victory! Judge Grants Preliminary Injunction in OPM/DOGE Lawsuit

4 days 18 hours ago
Court to Decide Scope of Injunction Later This Week

NEW YORK–In a victory for personal privacy, a New York federal district court judge today granted a preliminary injunction in a lawsuit challenging the U.S. Office of Personnel Management’s (OPM) disclosure of records to DOGE and its agents.

Judge Denise L. Cote of the U.S. District Court for the Southern District of New York found that OPM violated the Privacy Act and bypassed its established cybersecurity practices under the Administrative Procedures Act. The court will decide the scope of the injunction later this week. The plaintiffs have asked the court to halt DOGE agents’ access to OPM records and for DOGE and its agents to delete any records that have already been disclosed. OPM’s databases hold highly sensitive personal information about tens of millions of federal employees, retirees, and job applicants.

“The plaintiffs have shown that the defendants disclosed OPM records to individuals who had no legal right of access to those records,” Cote found. “In doing so, the defendants violated the Privacy Act and departed from cybersecurity standards that they are obligated to follow. This was a breach of law and of trust. Tens of millions of Americans depend on the Government to safeguard records that reveal their most private and sensitive affairs.”

The Electronic Frontier Foundation (EFF), Lex Lumina LLP, Democracy Defenders Fund, and The Chandra Law Firm requested the injunction as part of their ongoing lawsuit against OPM and DOGE on behalf of two labor unions and individual current and former government workers across the country. The lawsuit’s union plaintiffs are the American Federation of Government Employees AFL-CIO and the Association of Administrative Law Judges, International Federation of Professional and Technical Engineers Judicial Council 1 AFL-CIO

The lawsuit argues that OPM and OPM Acting Director Charles Ezell illegally disclosed personnel records to DOGE agents in violation of the Administrative Procedures Act and the federal Privacy Act of 1974, a watershed anti-surveillance statute that prevents the federal government from abusing our personal information. In addition to seeking to permanently halt the disclosure of further OPM data to DOGE, the lawsuit asks for the deletion of any data previously disclosed by OPM to DOGE.

The federal government is the nation’s largest employer, and the records held by OPM represent one of the largest collections of sensitive personal data in the country. In addition to personally identifiable information such as names, social security numbers, and demographic data, these records include work information like salaries and union activities; personal health records and information regarding life insurance and health benefits; financial information like death benefit designations and savings programs; nondisclosure agreements; and information concerning family members and other third parties referenced in background checks and health records.

OPM holds these records for tens of millions of Americans, including current and former federal workers and those who have applied for federal jobs. OPM has a history of privacy violations—an OPM breach in 2015 exposed the personal information of 22.1 million people—and its recent actions make its systems less secure. 

With few exceptions, the Privacy Act limits the disclosure of federally maintained sensitive records on individuals without the consent of the individuals whose data is being shared. It protects all Americans from harms caused by government stockpiling of our personal data. This law was enacted in 1974, the last time Congress acted to limit the data collection and surveillance powers of an out-of-control President. 

A number of courts have already found that DOGE’s activities at other agencies likely violate the law, including at the Social Security Administration and the Treasury Department.

For the preliminary injunction: https://www.eff.org/document/afge-v-opm-opinion-and-order-granting-preliminary-injunction
For the complaint: https://www.eff.org/document/afge-v-opm-complaint
For more about the case: https://www.eff.org/cases/american-federation-government-employees-v-us-office-personnel-management

Contacts:
Electronic Frontier Foundation: press@eff.org
Lex Lumina LLP: Managing Partner Rhett Millsaps, rhett@lex-lumina.com

Hudson Hongo

[B] 差別やデマを助長する恐れのある「不法滞在者ゼロプラン」 市民有志がプランの再考を求める意見書を公表

4 days 19 hours ago
政府は先月、国内の不法滞在者ゼロを目指す「不法滞在者ゼロプラン」を発表。鈴木法相は記者会見で「ルールを守る外国人を積極的に受け入れる一方で、守らない外国人は速やかに国外退去させる。将来的には不法滞在者ゼロをしっかりと目指していく」と語った。ところが、政府が示す「ルール」の線引きは曖昧なものであり、「政府の裁量次第で本来保護されるべき難民が国外退去の対象になるのでは」といった懸念の声も聞かれる。(藤ヶ谷魁)
日刊ベリタ

【横浜市再開発】「旧庁舎、不当な安値」本人訴訟の原告招き例会=神奈川支部

4 days 21 hours ago
 歴史的な名建築の旧市庁舎を一部取り壊し、超高層ビルを建設する横浜市の再開発事業を不当とする訴えが市民から起こされている。JCJ神奈川支部は、4月26日横浜市内で例会を開き、「横浜市民の財産を守る会」の高田尚暢代表に話を聞いた。 2020年6月に竣工した横浜市の新市庁舎は、高さ155m、32階の高層ビル。林文子前市長時代の2013年にそれまでの市庁舎から移転することが決定された。庁舎建設の傑作 横浜スタジアムに隣接し、JR関内駅前という好立地にある旧市庁舎は1959年に建てら..
JCJ

Contact

5 days 3 hours ago

To email us, replace [at] with @.

Email

  • General enquiries: office [at] statewatch.org
  • Send us an encrypted message via: secure [at] statewatch.org (PGP key)

To find out how to encrypt emails, you can use the guides here.

You can find individual staff members' contact details here.

Social media

Phone

  • +44 (0) 7836 296 043

Post

  • Statewatch, MDR, 88 Fleet Street, London EC4Y 1DH

Contribute material to Statewatch

We welcome submissions of material for publication on our website. See here for more information on contributing work to Statewatch.

justin