Machine Learning

In this lecture, we will look at formal models of learnability. We have already seen one such model, namely the mistake bound model. Now, we will look Probably Approximately Correct learning.

Lecture slides

• The Theory of Generalization
  • Videos: [fall 2018], [fall 2017]
• An Analysis of Conjunction Learning
  • Videos: [fall 2018], [fall 2017]
• PAC learning: Definition
  • Videos: [fall 2018], [fall 2017]
• Occam’s Razor
  • Videos: [fall 2018], [fall 2017]
• Positive and negative learnability results
  • Videos: [fall 2018], [fall 2017]
• Agnostic Learning
  • Videos: [fall 2018], [fall 2017]
• Shattering and the VC dimension
  • Videos: [fall 2018], [fall 2017]

Links and Resources

• Chapter 10 of of Hal Daumé III, A Course in Machine Learning (available online)

• Chapter 7 of Tom Mitchell’s book

• Chapter 6 of Hopcroft and Kannan’s Foundations of Data Science (available online)

• Chapters 3, 4, 6 of Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms (Available online)

Additional resources

• Chernoff-Hoeffding Inequality

• A proof of Chernoff’s inequality