Search

Visual abductive reasoning is the logical process of drawing the most plausible hypothesis based on given observations. This ability is fundamental to artificial intelligence because it enables inference from incomplete information. However, little research has been conducted on Korean visual abductive reasoning. To examine the capability of a multimodal language model’s Korean visual abductive reasoning, we set a simple baseline model and analyzed how it numerically estimated the plausibility for all Korean hypothesis sentences through a multiple-choice task. This task was implemented using a simple dual encoder model and the Korean Story Cloze dataset. After fine-tuning with the binary-choice task discriminating the plausible hypothesis from the implausible one, our baseline model shows an accuracy of 79.81%. In multiple-choice task designed to check for the influence of overfitting or annotation artifacts, the model estimated the plausibilities of four options in the order of Groundtruth≃ Plausible>Implausible≫Random. We also conducted experiments to analyze how the model performed Korean visual abductive reasoning. It was observed that the model made little use of the observation before the hypothesis but demonstrated a similar tendency to humans, struggling with data samples which humans also struggle with when evaluating the plausibility of given sentences. Our study sets a research foundation for numerically analyzing and understanding the language models’ visual abductive reasoning ability in the Korean context. It also shows both the potential and limitations of the language model’s Korean visual abductive reasoning ability and provides clues for future research directions. [ABSTRACT FROM AUTHOR]