Contents Science Lab

Nagoya University Graduate School of Informatics
Back to project list

Project: Imageability estimation

The semantic gap is the lack of coincidence between the information one can extract from data and its interpretation. For a machine it is often challenging to select the best fitting wording out of a group of candidates: For an image of a car, the tag “Vehicle” might be too vague, while the model number might be too specific. An understanding of the visual implications of tag can help to decide the correct degree of abstractness.

In this research, we estimate a measurement for the perceived differences between words. Abstract words have a broad mental image due to them being less visually defined, while concrete words with a rather narrow visual feature space are visually easier to grasp. We cooporate this idea into the model and look for feature variety differences of large datasets for different words.

Comparing various modalities like vision, language, and phonemes, we explore differences of perception regarding words when it comes to each modality.

[Sources and pretrained model] https://github.com/mkasu/imageabilityestimation

[Our results and dataset] https://github.com/mkasu/imageabilitycorpus

Featured image

Main project members

Avatar image
Chihaya Matsuhira

D3 student

Avatar image
Dr. Marc A. Kastner

Cooperative Research Fellow (Hiroshima City University)

Recent publications

    A multi-modal dataset for analyzing the imageability of concepts across modalities
    Marc A. Kastner, Chihaya Matsuhira, Ichiro Ide, Shin'ichi Satoh
    Proc. 4th IEEE Int. Conf. on Multimedia Information Processing and Retrieval (MIPR2021), pp.213-218, Online, September 2021.
    Imageability estimation using visual and language features
    Chihaya Matsuhira, Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
    Proc ACM Int. Conf. on Multimedia Retrieval (ICMR) 2020, pp.306-310, Online, October 2020.
    Estimating the imageability of words by mining visual characteristics from crawled image data
    Marc Aurel Kastner, Ichiro Ide, Frank Nack, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
    Multimedia Tools and Applications, vol.79, no.25, pp.18167-18199, July 2020.
    On the quantification of the mental image of visual concepts for multi-modal applications
    Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
    IPSJ Tech. Rep. Computer Vision and Image Media, 2020-CVIM-222-5, Online, May 2020.
    TBA (in Japanese)
    Chihaya Matsuhira, Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
    26th ANLP Annual Meeting, no.P1-12; pp.45-48, Online, March 2020.
    Estimating the visual variety of concepts by referring to Web popularity
    Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
    Multimedia Tools and Applications, vol.78, no.7, pp.9463-9488, April 2019.
    A preliminary study on estimating word imageability labels using Web image data mining
    Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
    Proc. 25th ANLP Annual Meeting, no.A4-7; pp.747-750, Nagoya Univ, March 2019.
    TBA (in Japanese)
    Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase
    IPSJ Tech. Rep. Computer Vision and Image Media, 2018-CVIM-211-4, Ritsumeikan Univ., Biwako-Kusatsu Campus, March 2018.

Last updated: 2024-08-02 00:26:53.664726847 +0000 UTC m=+1.646637966.