In this lecture, we will take a look at neural networks. First, we will see what neural networks are from the structural point of view and as a hypothesis class. Then we will look at the problem of training the parameters of a neural network using backpropagation.
Note that neural networks represent a vast collection of topics and the lecture only covers a very small slice. Students are encouraged to read the references if they are interested in learning more about this topic.
Lecture Slides
 Videos:
 Introduction and forward pass
 Backpropagation
 Practical concerns
 Older videos:
Links and Resources

Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville: An online book that focuses on deep neural networks. Part II of the book goes into depth about deep neural networks.

Deep Learning in Neural Networks: An Overview by Juergen Schmidhuber

Chapter 20 of Understanding Machine Learning: From Theory to Algorithms by Shai ShalevShwartz and Shai BenDavid (available online)

A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg