Improved Image Caption Rating – Datasets, Game, and Model

Conference Paper

Abstract

How well a caption fits an image can be difficult to assess due to the subjective nature of caption quality. What is a good caption? We investigate this problem by focusing on image-caption ratings and by generating high quality datasets from human feedback with gamification. We validate the datasets by showing a higher level of inter-rater agreement, and by using them to train custom machine learning models to predict new ratings. Our approach outperforms previous metrics – the resulting datasets are more easily learned and are of higher quality than other currently available datasets for

Conference Name

2023 CHI Conference on Human Factors in Computing Systems

Conference Location

Hamburg, Germany

Year of Publication

2023

Date Published

04/2023

Publisher

Association for Computing Machinery