Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?

Overview

• Vision and language models (VLMs) are currently the most capable architectures for multimodal tasks, which involve processing both visual and textual information.
• VLMs can not only make predictions, but also generate explanations for their outputs, either after the fact (post-hoc) or as part of the reasoning process (chain-of-thought).
• However, it is unclear how much VLMs actually rely on the visual and textual modalities when generating predictions versus explanations.

Plain English Explanation

Vision and language models (VLMs) are powerful AI systems that can handle tasks involving both images and text. They can not only make predictions, but also explain their reasoning. However, the paper investigates whether VLMs use the visual and textual information differently when generating predictions versus explanations.

The researchers wanted to understand if VLMs rely more on the image or the text when they are producing an answer versus when they are explaining their answer. They also looked at how consistent the VLMs are with themselves when generating these explanations, compared to language-only models.

Technical Explanation

The paper examines how vision and language models (VLMs) use the visual and textual modalities when generating predictions versus explanations. They evaluate the self-consistency of VLM decoders in both post-hoc and chain-of-thought (CoT) explanation settings, extending existing tests and measures.

The key findings are:

• VLMs are less self-consistent than language-only models (LLMs) in their explanations.
• The textual contributions in VLM decoders are much larger than the visual contributions across tasks.
• The visual contributions are significantly larger when generating explanations compared to generating answers.
• This difference in visual reliance is even greater in the CoT explanation setting versus the post-hoc setting.

The paper also provides an updated benchmark of state-of-the-art VL decoders on the VALSE benchmark, which has traditionally focused on VL encoders. The results show that VL decoders are still struggling with many of the phenomena tested by VALSE.

Critical Analysis

The paper raises important questions about the inner workings of VLMs and how they utilize visual and textual information differently for making predictions versus generating explanations. The finding that VLMs are less self-consistent than LLMs in their explanations is noteworthy and suggests potential issues with the robustness and reliability of these models.

The observation that VLMs rely much more heavily on textual inputs than visual inputs, even when generating explanations, is somewhat surprising given the models' multimodal nature. This raises questions about the ability of these models to truly integrate visual and linguistic information.

Additionally, the increased reliance on visual inputs for explanations compared to predictions, especially in the CoT setting, suggests that VLMs may be using the visual information in a more meaningful way when generating explanations. However, the paper does not delve deeply into the reasons behind this difference.

Further research is needed to better understand the underlying mechanisms and decision-making processes of VLMs, as well as to explore ways to improve the transparency and interpretability of these powerful models.

Conclusion

This paper provides valuable insights into the inner workings of vision and language models (VLMs), revealing that these models rely on visual and textual information differently when generating predictions versus generating explanations. The finding that VLMs are less self-consistent than language-only models in their explanations is concerning and suggests potential issues with the reliability of these models.

The research also highlights the need for continued efforts to improve the transparency and interpretability of large multimodal models, which are becoming increasingly influential in various applications. By understanding how VLMs process and integrate information from different modalities, we can work towards developing more robust and trustworthy AI systems.