The Alignment Problem book club questions

1. 1.

   Christian distinguishes between the reward signal a system is given and the goal its designers actually intend. Can you think of an example from your own work where measuring the wrong thing produced perverse results?

2. 2.

   Reward hacking examples range from silly (a boat racer spinning in circles) to serious (a hiring algorithm screening out qualified candidates). At what point does the same underlying failure become a moral problem rather than just a technical one?

3. 3.

   The book shows that training data encodes historical biases. Who should be responsible for identifying and correcting those biases — the engineers who build the system, the companies that deploy it, or the regulators who oversee it?

4. 4.

   Interpretability research wants to understand what neural networks have learned. Do you think it matters whether we can explain why an AI system works, or is reliable performance sufficient justification for deployment?

5. 5.

   Inverse reinforcement learning infers human values from behavior rather than asking people to articulate them. What does it mean to use behavior as a proxy for values, given how often people behave inconsistently with what they say they value?

6. 6.

   Cooperative AI proposes systems that defer to humans under uncertainty. Is that a satisfying solution, or does it just move the problem — now you need to trust that humans will make good decisions when deferred to?

7. 7.

   The alignment problem in AI has a structural resemblance to management problems in human organizations: how do you ensure that agents act in line with principals' actual goals? What can each domain learn from the other?

8. 8.

   Christian covers researchers who believe alignment is an existential risk and researchers who think the concerns are overblown. After reading the book, where do you find yourself, and what changed your view if anything?

9. 9.

   Several examples in the book involve systems deployed in criminal justice and hiring. What level of accuracy or fairness should be required before an AI system can make decisions affecting people's lives?

10. 10.

    The book was published in 2020. Which of the concerns Christian raises have since become more urgent, and which have been addressed by the progress in the field?

11. 11.

    If you were designing an AI system for a consequential application — medical diagnosis, say — what safeguards would you insist on before deployment, given what Christian describes?

How to run the discussion

• Pick six to eight, not all of them. An hour of real discussion covers fewer questions than anyone expects, and racing the list is what makes a book club feel like homework.
• Open with what people noticed, not what they concluded. Asking for a verdict first hardens opinions before anyone has heard each other.
• Let the disagreement run. The questions below that split a room are the ones worth the time; the ones everyone answers the same way can be skipped.
• Ask the quiet person a specific question. A direct, narrow question is far easier to answer than an open invitation to contribute.