We investigate the extent to which popular recurrent and attention-based neural models trained with an auto-regressive language modeling objective can represent the hierarchical nature of language by probing linguistic and structural properties using diagnostic classifiers. We tune our classifiers to be selective, for which we introduce a novel tree distance control task and the generalized selectivity metric. Our results suggest that access to bidirectional context improves the quality of extracted parse tree geometry.
- Michael J. Neely
- Vanessa Botha
This is a student project from the University of Amsterdam's Natural Language Processing 2, Spring 2020 course. It requires access to some code and data located here.