CS690: Natural Language Processing
Spring 2011

---

Time and Location: Tue, Thu 1:10pm – 3:00pm, Irvine 110
Instructor: Razvan Bunescu
Office: Stocker 337
Office Hours: Tue, Thu 10:30am – 12pm, or by email appointment
Email: bunescu @ ohio edu

Textbook:

Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, by Daniel Juraksfy and James E. Martin. McGraw Hill, 2008 (2nd edition).

Recommended Supplementary Text:

Foundations of Statistical Natural Language Processing by Christopher Manning and Hinrich Schütze. MIT Press, 1999.

Course description:
Natural Language Processing (NLP) is a branch of Artificial Intelligence concerned with developing computer systems that can process, understand, or communicate in natural language. Major applications of NLP include information retrieval and web search, information extraction, question answering, machine translation, sentiment analysis, text mining, and speech recognition. This graduate level course will give a fairly broad overview of NLP, with a primary focus on tasks that are widely seen as fundamental for a natural language understanding system such as part of speech tagging, syntactic parsing, word sense disambiguation, semantic role labeling, coreference resolution, and semantic parsing.

Prerequisites:
Students are expected to exhibit basic knowledge of formal languages (regular and context free grammars) and to be comfortable with programming in Java. Relevant background material in elementary linear algebra, probability theory and information theory will be made available during the course. Knowledge of machine learning and linguistics will be very useful, though not strictly necessary.

Lecture notes:

1. Syllabus & Introduction
2. Language Models
   • An Empirical Study of Smoothing Techniques for Language Modeling, Stanley F. Chen and. Joshua Goodman, Technical Report, 1998
   • All Our N-gram are Belong to You, Alex Franz and Thorsten Brantz (All your base are belong to us, Zero Wing)
3. Part of Speech Tagging
   • Part-of-Speech Tagging Guidelines for the Penn Treebank Project, Beatrice Santorini, Technical Report, 1990
   • A Maximum Entropy Model for Part-Of-Speech Tagging, Adwait Ratnaparkhi, EMNLP, 1996
   • Hidden Markov Models
4. Syntax and Grammars
   • Bracketing Guidelines for the Penn Treebank Project, Bies et al., Technical Report, 1995
   • Generating typed dependency parses from phrase structure parses, De Marneffe et al., LREC, 2006
   • Combinatory Categorial Grammar, Mark Steedman and Jason Baldridge, Technical Report, 2003
5. Syntactic Parsing
6. Statistical Parsing
   • Head-Driven Statistical Models for Natural Language Processing, Michael Collins, Computational Linguistics, 2003
   • Non-projective dependency parsing using spanning tree algorithms, McDonald et al., HLT-NAACL, 2005
7. Lexical Semantics and Word Sense Disambiguation
8. Semantic Role Labeling

Homework Assignments:

• Assignment 1
  • Language and Task Independent Text Categorization with Simple Language Models, Fuchun Peng, Dale Schuurmans and Shaojun Wang, HLT-NAACL 2003
• Assignment 2
• Assignment 3
  • Accurate Methods for the Statistics of Surprise and Coincidence, Ted Dunning, Computational Linguistics, 1993
• Assignment 4

Final Project:

• Project suggestions
• Project report guidelines

Online resources:

• NLTK
• Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze. Cambridge University Press, 2008.
• James H. Martin's introduction to probabilities
• Jason Eisner's equestrian introduction to probabilities