The Master Algorithm, in detail
The Master Algorithm is Pedro Domingos's survey of machine learning — the field of computer science that creates algorithms capable of learning from data — organized around a central speculative thesis: that there exists, or may be found, a single master algorithm from which all learning can be derived. Domingos argues that the major schools of machine learning are each working on different facets of the same underlying problem, and that a unified learner that combines their strengths would be capable of deriving any knowledge from data.
The five tribes of machine learning that Domingos describes each embody a different metaphor for learning. Symbolists derive rules from examples using inverse deduction, following in the tradition of logic and formal systems. Connectionists build systems inspired by neural structure, most powerfully expressed in modern deep learning. Evolutionists use simulated evolution — genetic algorithms — to discover solutions by survival of the fittest. Bayesians reason probabilistically under uncertainty, updating beliefs as evidence accumulates. Analogizers generalize from similar examples, expressed most powerfully in support vector machines and k-nearest-neighbor approaches.
The second half of the book traces the applications of machine learning in search engines, recommendation systems, medical diagnosis, personalized advertising, and scientific discovery. Domingos is enthusiastic about the field's potential to solve problems in biology, physics, and economics that are too complex for human mathematical analysis but tractable for pattern recognition at scale.
The master algorithm idea itself — that all learning can be unified — is speculative and has not been achieved. Critics point out that different learning problems genuinely require different approaches and that the unified-learner vision may be an overreach. The book's greatest value is its accessible survey of the five major approaches and their applications, not its grand unified theory.
The big ideas
- 1.
Machine learning is the programming paradigm that creates algorithms which improve their performance through experience rather than being explicitly programmed to solve specific problems.
- 2.
Five major schools of machine learning — symbolists, connectionists, evolutionists, Bayesians, and analogizers — represent different approaches to the same fundamental problem of learning from data.
- 3.
Deep learning, the most powerful current form of connectionist learning, builds hierarchical representations of data through many layers of artificial neurons, and has achieved human-level or superhuman performance on vision and language tasks.