A preliminary study on estimating word imageability labels using Web image data mining
Back to publications
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
Natural Language Processing (NLP) is a key element in many real-world applications. However, the semantic gap is an ongoing problem, leading to unnatural results disconnected from the users. To understand semantics in real-world scenarios, human perception needs to be taken into consideration. Imageability is an approach to quantize human perception of words. Research shows a relationship between language usage and the imageability of words, making it useful for multimodal applications. However, imageability datasets are typically created by hand. In this research, a method using image data mining to estimate the imageability of words is proposed. The main assumption is a relationship between the imageability of concepts and crowd-sourced images. We use visual features from Web-crawled images to train a model to predict imageability. The model is evaluated using a test dataset. The proposed method can be used to increase the corpus of imageability dictionaries.
Type: Talk at 25th Annual Meeting of the Association for Natural Language Processing (言語処理学会第25回年次大会)
Date: March 2019