Cathy O'Neil

Cathy O'Neil is a mathematician, data scientist, author, and algorithmic auditor whose work has been central to the public discourse on the harms of automated decision-making systems. She holds a PhD in mathematics from Harvard University and spent the early part of her career in academia before moving to the hedge fund industry, where she worked as a quantitative analyst. Her experience building financial models during and after the 2008 financial crisis was formative: she saw firsthand how mathematical models presented as objective were embedded with assumptions and incentives that produced catastrophic outcomes.

That experience led her to shift her focus toward the social consequences of algorithmic systems. She became a data scientist and founded ORCAA, an algorithmic auditing company that evaluates AI and algorithmic systems for bias, fairness, and unintended consequences. She is also a regular contributor to Bloomberg Opinion, where she has written on AI bias, financial models, and tech industry accountability.

Her 2016 book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy became one of the most widely cited works in the AI ethics and accountability literature. The book introduced the term "WMD" as a framework for identifying dangerous algorithms, those that are opaque, widely scaled, and have damaging feedback loops, and documented cases across hiring, credit scoring, criminal justice, education, and insurance where algorithmic systems were systematically amplifying existing inequalities while appearing neutral and objective.

O'Neil was awarded a MacArthur Fellowship (commonly known as a "genius grant") in 2019, recognizing her contributions to algorithmic accountability.

Through ORCAA, O'Neil has conducted algorithmic audits across industries including hiring, insurance, education, and consumer credit, working with companies, regulators, and civil society organizations to translate the broad concept of algorithmic accountability into specific, testable assessments of real systems. That auditing work has informed regulatory developments in jurisdictions including New York City, where ORCAA's research contributed to the design of automated employment decision tool regulations. Her ongoing public writing through Bloomberg Opinion and her long-running blog Mathbabe has continued to push the same fundamental argument that Weapons of Math Destruction introduced: when an algorithm makes consequential decisions about people at scale, the absence of an audit and the absence of recourse are not neutral defaults but design choices with social consequences.

Cathy O'Neil's Weapons of Math Destruction is cited directly in the treatise as primary evidence for the first AI First Principle: AI Inherits Messiness. The treatise quotes her central finding precisely: "when biased data trains predictive models, those models don't just reflect existing inequities — they amplify and legitimize them under the guise of algorithmic neutrality."

Her work changed how the AI field understood the relationship between training data and output bias. Before Weapons of Math Destruction, algorithmic bias was treated as a technical error to be patched. O'Neil reframed it as a structural property of how machine learning works: AI trained on human decisions inherits the assumptions, prejudices, and contextual constraints of those decisions, and then executes them at machine speed and scale. That reframing is the foundation of the first principle.