Abstract Gijs Wijnhols 15 October 2018

From IMC wiki
Jump to: navigation, search

Title: "Non-Linear Composition in Distributional Semantics: Modelling Ellipsis, Anaphora and Structural Ambiguities"

Abstract:

Distributional semantics represents word meaning as a vector by considering how words co-occur in big text corpora. Such embeddings allow one to easily compare word meanings by means of a distance measure. Another advantage of these embeddings is that they allow for a compact representation that can be used as input to a Neural Network (where one-hot encodings for instance would be ineffective).

Compositional distributional semantics tries to approximate the meaning of bigger phrases, sentences, or even documents, in a similar vectorial form. Here we cannot rely on distributional information because of (a) the compositional nature of language and (b) data sparsity. So we need to come up with techniques to create these phrase, sentence or document embeddings.

A very broad dichotomy of compositional distributional models distinguishes between (a) formal models, deriving the embedding for the sentence from the individual word embeddings combined with some formal structure, and (b) neural models, which learn a map from the individual word embeddings to some sentence embedding using one or more NLP tasks to achieve this.

In this talk, I will present my PhD work so far, which falls under class (a). I will start out with a discussion of a formal model that links grammar formalisms to vectors and linear maps. Then, I will illustrate the need for non-linear models to deal with ellipsis and anaphora, and present/discuss such a model for verb phrase ellipsis and anaphora. I will give experimental results on newly developed sentence similarity datasets, and discuss a couple of future lines for research.