Machine Learning

In this lecture, we will take a look at neural networks. First, we will see what neural networks are from the structural point of view and as a hypothesis class. Then we will look at the problem of training the parameters of a neural network using backpropagation.

Note that neural networks represent a vast collection of topics and the lecture only covers a very small slice. Students are encouraged to read the references if they are interested in learning more about this topic.

Lecture Slides

• Neural Networks: Introduction

• Prediction: The Forward Pass

• Backpropagation

• Practical Concerns

• Videos:
  • Introduction and forward pass
  • Backpropagation
  • Practical concerns
• Older videos:
  • Neural networks (1/2)
  • Neural networks (2/2)

Links and Resources

• Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville: An online book that focuses on deep neural networks. Part II of the book goes into depth about deep neural networks.

• Deep Learning in Neural Networks: An Overview by Juergen Schmidhuber

• Chapter 20 of Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David (available online)

• A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg