In this lecture, we will look at formal models of learnability. We have already seen one such model, namely the mistake bound model. Now, we will look Probably Approximately Correct learning.
Lecture slides
- The Theory of Generalization
- An Analysis of Conjunction Learning
- PAC learning: Definition
- Occam’s Razor
- Positive and negative learnability results
- Agnostic Learning
- Shattering and the VC dimension
Links and Resources
-
Chapter 10 of of Hal Daumé III, A Course in Machine Learning (available online)
-
Chapter 7 of Tom Mitchell’s book
-
Chapter 6 of Hopcroft and Kannan’s Foundations of Data Science (available online)
-
Chapters 3, 4, 6 of Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms (Available online)