Seminar in Computational Linguistics
- Date: –15:00
- Location: Engelska parken 9-3042
- Lecturer: Paola Merlo
- Contact person: Miryam de Lhoneux
Probing word and sentence embeddings for long-distance dependencies effects in French and English
Despite their practical success and impressive performances, neural-network-based and distributed semantics techniques have often been criticized as they remain fundamentally opaque and difficult to interpret. Several recent pieces of work have investigated the linguistic abilities of these representations, and whether they can capture long agreement and thus hierarchical notions. Results are at present inconclusive. In this vein, we study another core, defining and more challenging property of language: the ability to construe long-distance dependencies. Human languages exhibit the ability to interpret discontinuous elements distant from each other in the string as if they were adjacent. This ability is blocked if a similar, but extraneous, element intervenes between the discontinuous components. We present results that show that word embeddings and the similarity spaces they define do not unequivocally correlate with experimental results on intervention similarity in long-distance dependencies narrowly defined, even if translated in more 'syntactic' portion of the distributional space or tested in prediction tasks. These results show that the linguistic encoding in distributed representations does not appear to be human-like, and it also brings evidence to the debate on narrow or broad definitions of similarity in syntax and sentence processing.
Paola Merlo is associate professor in the Linguistics department of the University of Geneva. She is the head of the interdisciplinary research group Computational Learning and Computational Linguistics (CLCL). The group is concerned with interdisciplinary research combining linguistic modelling with machine learning techniques. Prof. Merlo has been editor of Computational Linguistics, published by MIT Press and a member of the executive committee of the ACL. Prof. Merlo holds a doctorate in Computational Linguistics from the University of Maryland, USA. She has been associate research fellow at the University of Pennsylvania, and visiting scholar at Rutgers, Edinburgh, and Stanford.