Conditional Structure versus Conditional Estimation in NLP Models

Abstract

This paper separates conditional parameter estimation, which consistently raises test set accuracy on statistical NLP tasks, from conditional model structures}, such as the conditional Markov model used for maximum-entropy tagging, which tend to lower accuracy. Error analysis on the POS tagging task shows that the actual tagging errors made by the conditionally structured model derive principally not from label bias, as has been claimed, but from other ways in which the independence assumptions of the conditional model structure are unsuited to linguistic sequences. The paper presents new word-sense disambiguation and POS tagging experiments, and integrates apparently conflicting reports from other recent work.