The Book of Why, in detail
The Book of Why is Judea Pearl's argument that the dominant tradition in statistics — which insists on correlations and avoids causal claims — is a fundamental intellectual mistake, and that building a proper science of causality is the most important unsolved problem in both science and artificial intelligence. Pearl is a computer scientist and winner of the Turing Award who spent decades developing the mathematical framework known as causal inference, and this book is his accessible account of that work, written with science journalist Dana Mackenzie.
Pearl organizes his argument around what he calls the "ladder of causation." The first rung is association: seeing, observing, asking what goes with what. The second is intervention: doing, asking what happens if I act. The third is counterfactual: imagining, asking what would have happened if things had been different. Standard statistics, Pearl argues, lives almost entirely on the first rung. It can identify correlations with great precision but cannot tell you whether smoking causes cancer, whether a drug causes recovery, or what would have happened had you taken a different path. To answer causal questions, you need causal tools.
The book traces the intellectual history of this problem with surprising drama. Pearl describes the debates between statisticians like Karl Pearson and Francis Galton, who built the entire edifice of modern statistics on the deliberate rejection of causal language, and the scientists — Sewall Wright in genetics, Phillip Wright in economics — who kept trying to sneak causality back in. The core technical contribution Pearl explains is the do-calculus and causal diagrams, tools that let you formally represent causal structures and derive what can be learned from observational data versus what requires intervention.
The implications for artificial intelligence are Pearl's most ambitious claim. Current machine learning systems, however powerful, operate on the first rung of the ladder. They identify patterns in data extraordinarily well, but they cannot reason about interventions or counterfactuals. Pearl argues this means current AI cannot reason the way humans do, and that building truly intelligent systems will require giving machines a causal model of the world — not just a statistical one. The book is partly a history, partly a technical tutorial, and partly a manifesto for a research agenda Pearl believes most of his field has been too timid to pursue.
The big ideas
- 1.
The ladder of causation has three rungs: association (seeing), intervention (doing), and counterfactual (imagining). Standard statistics is confined to the first rung.
- 2.
Correlation cannot establish causation — this is widely known. Pearl's contribution is providing the mathematical tools to establish causation from observational data under specified conditions.
- 3.
Causal diagrams (directed acyclic graphs) let researchers explicitly represent their assumptions about causal structure, making those assumptions visible and testable rather than buried in methodology.