This lecture covers various broad families that we can use for predicting structured output. In particular, we will cover graph based algorithms, heuristics such as beam search, Monte Carlo methods like Gibbs sampling and the use of integer linear programming for inference.
Lectures
Readings
- Chapter 2 of Noah Smith, Linguistic Structure Prediction
Inference with Dynamic Programming
- (*) Täckström, Oscar, Kuzman Ganchev, and Dipanjan Das. Efficient inference and structured learning for semantic role labeling. Transactions of the Association for Computational Linguistics 3 (2015): 29-41.
Non-exact inference
-
(*) Jenny Rose Finkel, Trond Grenager, and Christopher Manning, Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling, ACL 2005
-
(*) Alexander Rush and Michael Collins, A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing, JAIR 2012.
-
(*) Sameer Singh, Michael Wick and Andrew McCallum, Monte Carlo MCMC: Efficient Inference by Approximate Sampling, EMNLP 2012.
Integer programming for inference
-
(*) Dan Roth and Wen-tau Yih, Integer Linear Programming Inference for Conditional Random Fields, ICML 2005.
-
(*) Dan Roth and Wen-tau Yih, Global Inference for Entity and Relation Identification via a Linear Programming Formulation. Introduction to Statistical Relational Learning, 2007.
-
(*) Sebastian Riedel and James Clarke, Incremental Integer Linear Programming for Non-projective Dependency Parsing, EMNLP 2006.
-
(*) Berant, Jonathan, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang, Peter Clark, and Christopher D. Manning. Modeling Biological Processes for Reading Comprehension. In EMNLP. 2014.
Learning to search
-
(*) Hal Daume, J. Langford, and Daniel Marcu, Search-based Structured Prediction, Machine Learning 2009.
-
(*) Chang, Kai-Wei, Akshay Krishnamurthy, Alekh Agarwal, Hal Daume, and John Langford. Learning to Search Better than Your Teacher In International Conference on Machine Learning, pp. 2058-2066. 2015.