Deep Learning book club questions

1. 1.

   The book distinguishes between what deep learning can do and why it works theoretically. Does that gap between empirical success and theoretical understanding concern you, or is it an acceptable state for an engineering discipline?

2. 2.

   Backpropagation was known for decades before it became practically useful. What conditions enabled the deep learning revolution of the 2010s beyond the algorithm itself?

3. 3.

   Convolutional networks are designed around assumptions about spatial structure in images. What does it mean that a useful architecture requires encoding human assumptions about the structure of the world?

4. 4.

   The book was written in 2016. Which of its frontier topics — GANs, attention mechanisms, transfer learning — have since become standard, and which remain open problems?

5. 5.

   Deep learning requires enormous datasets for training. What are the implications of that dependency for who can build useful AI systems and who cannot?

6. 6.

   The book covers techniques like dropout and weight decay that prevent memorization. What does the existence of these techniques reveal about what neural networks would otherwise do?

7. 7.

   A recurrent network can in principle handle sequences of arbitrary length, but training them on long sequences is notoriously difficult. What does this limitation tell us about the difference between what a system can theoretically do and what it can practically learn?

8. 8.

   Bengio, Goodfellow, and Courville are researchers at the frontier of the field writing about their own work. Where does the book feel like advocacy for their approaches, and where does it feel like neutral exposition?

9. 9.

   Deep learning has produced systems that can detect cancer in medical images, write coherent text, and generate realistic faces. Does the quality of the output change your intuitions about whether the system is doing something meaningfully intelligent?

10. 10.

    The book is available for free online while also published as a textbook. What does that decision say about the incentive structures in academic research and technical publishing?

11. 11.

    Deep learning has been criticized for being a black box — effective but unexplainable. After reading this book, do you think that opacity is fundamental to how these systems work, or an engineering problem that will be solved?

12. 12.

    What would a reader need to do after finishing this book to be capable of doing original deep learning research, and how does that gap illustrate the difference between technical literacy and research fluency?

How to run the discussion

• Pick six to eight, not all of them. An hour of real discussion covers fewer questions than anyone expects, and racing the list is what makes a book club feel like homework.
• Open with what people noticed, not what they concluded. Asking for a verdict first hardens opinions before anyone has heard each other.
• Let the disagreement run. The questions below that split a room are the ones worth the time; the ones everyone answers the same way can be skipped.
• Ask the quiet person a specific question. A direct, narrow question is far easier to answer than an open invitation to contribute.