In this lecture, we will take a look at the widely used Perceptron algorithm and its associated mistake bound theorem.
Lecture slides
 The Perceptron algorithm and its variants
 Videos (covering both the algorithm and the mistake bound theorem): [fall 2018, part 1], [fall 2018, part 2], [fall 2017, part 1], [fall 2017, part 2]
Links and Resources

Chapters 3 and 6 of Hal Daumé III, A Course in Machine Learning

Dan Roth, OnLine Learning of Linear Functions (course notes)
Further reading

Freund, Yoav, and Robert E. Schapire. Large margin classification using the perceptron algorithm. Machine learning 37, no. 3 (1999): 277296.

Roni Khardon and Gabriel Wachman, Noise Tolerant Variants of the Perceptron Algorithm, Journal of Machine Learning Research , Vol 8, pp 227–248, 2007