Dr. Marc A. Kastner

Cooperative Research Fellow （Hiroshima City University）

E-Mail: mkastner ( a t ) nii.ac.jp

Personal website: https://www.marc-kastner.com

Dr. Marc A. Kastner received his BSc and MSc in Computer Science from Braunschweig University of Technology in Braunschweig, Germany, in 2013 and 2016, and his PhD in Informatics from the Graduate School of Informatics of Nagoya University, Japan in 2020. He is currently a postdoctoral researcher at the National Institute of Informatics, Japan. His research focuses on affective multimedia, language and vision and semantic gap problems.

Main projects

Book page colorization based on perception

In this research, we colorize pages of books according to human perception in order to summarize and compare book contents.

Image captioning considering imageability

In this research, we incorporate psycholinguistics into image captioning in order to tailor captions to different applications.

Imageability estimation

In this research, we model the human perception of the semantic gap for use in affective computing.

Recent publications

Interpolating the Text-to-Image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-Image Generation

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide
IEEE Access, Vol.12, pp.41299-41316, March 2024.

Computational measurement of perceived pointiness from pronunciation

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Takatsugu Hirayama, Yasutomo Kawanishi, Keisuke Doman, Daisuke Deguchi
Multimedia Tools and Applications, 83(9), pp.26183-26210, March 2024.

Correction to: Computational measurement of perceived pointiness from pronunciation

Image-Collection Summarization using Scene-Graph Generation with External Knowledge

PHUEAKSRI Itthisak, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide
IEEE Access, Vol.12, pp.17499-17512, January 2024.

An Approach to Generate a Caption for an Image Collection Using Scene Graph Generation

PHUEAKSRI Itthisak, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide
IEEE Access, Vol.11, pp.128245-128260, November 2023.

Nonword-to-Image Generation Considering Perceptual Association of Phonetically Similar Words

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Ichiro Ide
Proceedings of the 1st International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice, pp.115-125, October 2023.

Image Collection Scene Graph Summarization Enhancing Relation Predictor with External Knowledge

PHUEAKSRI Itthisak, Marc A. Kastner, 川西康友, Takahiro Komamizu, 井手一郎
第26回画像の認識・理解シンポジウム(MIRU2023), IS1-80, July 2023.

Leverage Semantic Alignment of Object Relations for Image Captioning

Da Huo, Marc A. Kastner, 平山高嗣, Takahiro Komamizu, 井手一郎
第26回画像の認識・理解シンポジウム(MIRU2023), IS1-85, July 2023.

MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results

Yuki Kondo, Norimichi Ukita, Takayuki Yamaguchi, Hao-Yu Hou, Mu-Yi Shen, Chia-Chi Hsu, En-Ming Huang, Yu-Chen Huang, Yu-Cheng Xia, Chien-Yao Wang, Chun-Yi Lee, Da Huo, Marc A. Kastner, Tingwei Liu, 川西康友, 平山高嗣, Takahiro Komamizu, 井手一郎, Yosuke Shinya, Xinyao Liu, Guang Liang, Syusuke Yasui
International Conference on Machine Vision Applications (MVA), TE-1, July 2023.

Small Object Detection for Birds with Swin Transformer

Da Huo, Marc A. Kastner, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide
International Conference on Machine Vision Applications (MVA), TE-3, July 2023.

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide
arXiv, cs.MM, arXiv:2303.03144, March 2023.

Toward Captioning an Image Collection from a Combined Scene Graph Representation Approach

PHUEAKSRI Itthisak, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide
29th International Conference on Multimedia Modeling (MMM2023), Vol.13833, pp.178-190, January 2023.

Detection of birds in a 3D environment referring to audio-visual information

Yasutomo Kawanishi, Ichiro Ide, Baidong Chu, Chihaya Matsuhira, Marc Aurel Kastner, Takahiro Komamizu, Daisuke Deguchi
Proc. 18th IEEE Int. Conf. on Advanced Video and Signal-based Surveillance (AVSS2022), December 2022.

Action Semantic Alignment for Image Captioning

Da Huo, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide
IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR), pp.194-197, August 2022.

On Estimating Evoked Emotions of Social Media Videos through User Comments Analysis

Yiming Wang, Marc A. Kastner, Takahiro Komamizu, Yasutomo Kawanishi, Takatsugu Hirayama, Ichiro Ide
Unknown Journal, IS3-72, July 2022.

Towards Captioning an Image Collection using Scene Graph

PHUEAKSRI Itthisak, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide
Unknown Journal, IS1-78, July 2022.

Towards Detecting Birds from Panorama Video Aided by Sound Source Localization

Baidong Chu, Chihaya Matsuhira, Yasutomo Kawanishi, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Daisuke Deguchi
IEICE Technical Report (MVE), 121(423), pp.60-64, March 2022.

Towards detecting birds from panorama video aided by Sound Source Localization

Baidong Chu, Chihaya Matsuhira, Yasutomo Kawanishi, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Daisuke Deguchi
IEICE Tech. Rep. Media Experience and Virtual Environment, March 2022.

Imageability- and Length-Controllable Image Captioning

Marc A. Kastner, Kazuki Umemura, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase, Shin’Ichi Satoh
IEEE Access, Vol.9, pp.162951-162961, November 2021.

Imageability- and Length-Controllable Image Captioning

Imageability- and length-controllable image captioning

Marc A. Kastner, Kazuki Umemura, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase, Shin'ichi Satoh
IEEE Access, November 2021.

A multi-modal dataset for analyzing the imageability of concepts across modalities

Marc A. Kastner, Chihaya Matsuhira, Ichiro Ide, Shin'ichi Satoh
2021 IEEE 4th International Conference on Multimedia Information Processing and Retrieval (MIPR), pp.213-218, September 2021.

A multi-modal dataset for analyzing the imageability of concepts across modalities

Marc A. Kastner, Chihaya Matsuhira, Ichiro Ide, Shin'ichi Satoh
Proc. 4th IEEE Int. Conf. on Multimedia Information Processing and Retrieval (MIPR2021), September 2021.

Pointedness of an Image: Measuring How Pointy an Image is Perceived

Chihaya Matsuhira, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
Proceedings of HCII 2021, Vol.37, pp.137-144, July 2021.

Introduction to the Contents Science Lab

Ichiro Ide, Keisuke Doman, Takahiro Komamizu, Marc A. Kastner, Hirotaka Kato, Chihaya Matsuhira
Joint International Workshop on Smart Systems, July 2021.

Introduction to the Contents Science Lab

Ichiro Ide, Keisuke Doman, Takahiro Komamizu, Marc A. Kastner, Hirotaka Kato, Chihaya Matsuhira
Joint Int. Workshop on Smart Systems, July 2021.

Pointedness of an image: Measuring how pointy an image is perceived

Chihaya Matsuhira, Marc Aurel Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
HCI International 2021 -Posters, 23rd HCI Int. Conf., HCII2021, Virtual Event, July 24-29, Procs., Part I, Constantine Stephanidis, Margherita Antona, Stavroula Ntoa eds., Communications in Computer and Information Science, July 2021.

Tell as you imagine: Sentence imageability-aware image captioning

Kazuki Umemura, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
27th Int. Conf. on Multimedia Modeling (MMM2021), Vol.2, pp.62-73, June 2021.

Tell as you imagine: Sentence imageability-aware image captioning

Kazuki Umemura, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
MultiMedia Modeling -27th Int. Conf., MMM2021, Prague, Czech Republic, June 22-24, 2021, Procs., Part II, Jakub Lokoč, Tomáš Skopal, Klaus Schoeffmann, Vasileios Mezaris, Xirong Li, Stefanos Vrochidis, Ioannis Patraseds., Lecture Notes in Computer Science, June 2021.

Tell as You Imagine: Sentence Imageability-aware Image Captioning

Kazuki Umemura, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase
Lecture Notes in Computer Science, Vol.12573, p.62–73, January 2021.