Abstract Dimitri Kartsaklis 5th December 2013
Compositional-distributional models of meaning aim to unify the two prominent semantic paradigms in natural language: the compositional perspective of formal semantics and the distributional models based on vector spaces. In this way, one can start from word vectors based on co-occurrence statistics collected by a corpus, and produce a vector that represents the meaning of a sentence. However, most of the current methods aiming to providing compositionality are based on ambiguous vector representations, where all the meanings of a polysemous word, such as "bank", are fused into the same vector or tensor. For many of the compositional models, this practice produces results that are hard to interpret. In this talk we investigate the effect of Word Sense Disambiguation in the context of compositionality in vector spaces, and we provide experimental evidence that explicitly separating the two tasks in distinct steps can provide better composite representations for a number of models.
The talk is roughly divided in three parts. In the first part we provide a short introduction to three generic classes of compositional-distributional models: vector mixtures, tensor-based models, and deep learning models. The second part explains how standard word sense induction and disambiguation techniques can be applied in compositional methods for vector spaces. The last part presents experimental results that evaluate the efficiency of the discussed methods in a variety of tasks and datasets, mainly based on sentence similarity.
The material we are going to discuss is based on the following works:
- Prior Disambiguation of Word Tensors for Constructing Sentence Vectors (Kartsaklis and Sadrzadeh). EMNLP 2013
- Separating Disambiguation from Compositional in Distributional Semantics (Kartsaklis, Sadrzadeh, Pulman). CoNLL 2013
- Combining Compositional and Distributional Models of Semantics (Pulman). Quantum Physics and Linguistics: A Compositional, Diagrammatic Discourse, Oxford University Press.
Back to Cognitive Science Seminar Series