"How Do Pronouns Affect Word Embedding" by Tonglee Chung, Bin Xu et al.

Article Title

Authors

Tonglee Chung, the Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
Bin Xu, the Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
Yongbin Liu, the School of Computer Science and Technology, University of South China, Hengyang 421001, China.
Juanzi Li, the Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
Chunping Ouyang, the School of Computer Science and Technology, University of South China, Hengyang 421001, China.

Keywords

word embedding, co-reference resolution, representation learning

Abstract

Word embedding has drawn a lot of attention due to its usefulness in many NLP tasks. So far a handful of neural-network based word embedding algorithms have been proposed without considering the effects of pronouns in the training corpus. In this paper, we propose using co-reference resolution to improve the word embedding by extracting better context. We evaluate four word embeddings with considerations of co-reference resolution and compare the quality of word embedding on the task of word analogy and word similarity on multiple data sets. Experiments show that by using co-reference resolution, the word embedding performance in the word analogy task can be improved by around 1.88%. We find that the words that are names of countries are affected the most, which is as expected.