The following is a tentative list (a superset, actually) of the topics to be covered in the class.

Note: This list will change as the semester progresses. For a listing of lectures, visit the schedule page.

  1. Introduction
    • What is a structure? Examples
    • What are the different computational challenges for structured prediction?
    • Modeling, training and inference
  2. Review of linear models for binary and multiclass classification
    • Linear models for binary classification
    • Perceptron, Logistic Regression and Support Vector Machine for binary classification
    • From binary classification to multiclass classification, models of multiclass classification
    • Loss functions, online vs batch learning
  3. Binary and Multiclass classification to structures
    • Simple extensions
    • Inference: Predicting multiple inter-related variables
    • Binary and multiclass classification as special structures
  4. Basic structured prediction models: Sequence labeling
    • Inference with classifiers
    • Hidden Markov Models
    • Conditional Random Fields for sequences
    • Structured Perceptron for sequences
    • Inference with sequences
  5. General conditional models
    • Connection to Markov Random Fields and Bayes Nets
    • Extension of structured perceptron and conditional random fields to general conditional models
    • Structured Support Vector Machines
    • Tradeoffs between different learning algorithms
  6. Inference in general conditional models
    • Combinatorial optimization
    • Dynamic programming
    • Framing inference as integer linear programs: Constrained conditional models
    • Modeling background knowledge
  7. Tradeoffs between different training regimes
    • Training with or without inference
    • Constraint driven learning, posterior regularization
  8. General recipe for fully supervised learning
  9. Learning without full supervision
    • Modeling latent variables in conditional models
    • Semi-supervised learning, indirect supervision and response-driven learning
  10. Dealing with computational complexity of inference
    • Approximate inference
    • Lagrangian relaxation and dual decomposition
  11. Structured Prediction and Deep Neural Networks
    • Structure, features and representations