The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling
0
💬
Sign in to get full access
Overview
- The researchers apply Conceptual Metaphor Theory (CMT) from cognitive linguistics to the visualization domain to study visual conceptual metaphors used in science infographics.
- Metaphors play an important role in visual communication, but their use is often based on intuition rather than a formal process.
- The researchers aim to develop a taxonomy and grammar to guide the creation of visual components like infographics.
- They analyze infographics from four domains (biomedicine, climate, space, and anthropology) to identify patterns in the use of visual conceptual metaphors.
Plain English Explanation
In this research, the team looked at how metaphors are used in scientific infographics to explain complex concepts. Metaphors are a common way to make complicated ideas more understandable by comparing them to something more familiar.
However, the use of metaphors in visualizations is often based on the creator's intuition rather than a formal process. The researchers wanted to develop a way to better understand and describe the metaphors used in infographics, so that they could be used more effectively.
To do this, the team analyzed a collection of infographics from different scientific domains, like medicine, climate, space, and anthropology. They broke down the visual components of these infographics and identified common patterns in the metaphors being used.
The analysis showed that two main types of metaphors are commonly used in scientific visualizations: ontological metaphors, which compare abstract concepts to physical objects, and orientational metaphors, which use spatial relationships to represent ideas.
By understanding these patterns, the researchers hope to develop a framework that can guide the creation of more effective and meaningful scientific infographics in the future.
Technical Explanation
The researchers apply an approach from cognitive linguistics, specifically Conceptual Metaphor Theory (CMT), to the visualization domain. Their goal is to address the patterns of visual conceptual metaphors that are commonly used in science infographics.
Metaphors play a crucial role in visual communication, as they are frequently employed to explain complex concepts. However, their use is often based on intuition rather than a formal process. The researchers aim to develop a taxonomy and grammar to guide the creation of visual components, such as infographics.
The researchers' classification of the visual conceptual mappings within scientific representations is based on a detailed analysis of data collected from four domains: biomedicine, climate, space, and anthropology. These domains represent a diverse range of visual conceptual metaphors used in the visual communication of science.
Through this analysis, the researchers are able to identify patterns in the use of visual conceptual metaphors, resolve ambiguities about why specific conceptual metaphors are used, and develop a better overall understanding of visual metaphor use in scientific infographics. The findings indicate that ontological and orientational conceptual metaphors are the most widely applied to translate complex scientific concepts.
To support their findings, the researchers developed a visual exploratory tool based on the collected database. This tool places the individual infographics on a spatio-temporal scale and illustrates the breakdown of visual conceptual metaphors.
Critical Analysis
The researchers provide a comprehensive analysis of the use of visual conceptual metaphors in scientific infographics, which is a valuable contribution to the field of visualization. By applying Conceptual Metaphor Theory from cognitive linguistics, they have developed a systematic approach to understanding and describing the metaphors used in these visualizations.
One potential limitation of the study is the relatively small sample size, as the analysis was based on infographics from only four scientific domains. While these domains represent a diverse range of topics, expanding the analysis to a broader set of infographics could provide further insights and strengthen the generalizability of the findings.
Additionally, the researchers acknowledge that their analysis is based on a breakdown of the visual components in existing infographics, which may not capture the full context and intent behind the use of specific metaphors. Further research could explore the decision-making processes of infographic creators and the intended impact of the metaphors they employ.
Despite these potential limitations, the researchers have made a valuable contribution to the understanding of visual metaphor use in scientific communication. Their work provides a foundation for developing tools and language to guide the creation of more effective and meaningful infographics, which could have important implications for science education, public engagement, and knowledge dissemination.
Conclusion
This research applies Conceptual Metaphor Theory from cognitive linguistics to the domain of data visualization, with a focus on the use of visual conceptual metaphors in scientific infographics. The researchers have developed a systematic approach to identifying and analyzing the patterns of metaphor use in these visualizations, which can inform the creation of more effective and meaningful infographics.
By examining infographics from diverse scientific domains, the researchers have found that ontological and orientational conceptual metaphors are the most widely used to translate complex scientific concepts. This understanding can help guide the development of a taxonomy and grammar for creating visual components that leverage metaphors in a more intentional and effective way.
The potential implications of this research extend beyond the realm of scientific communication, as the insights gained can also be applied to the design of visual metaphors in other domains, such as data visualization for code learning or visual metaphor captioning for videos. By bridging the gap between cognitive linguistics and data visualization, this work contributes to a deeper understanding of how metaphors can be leveraged to enhance the communication of complex ideas through visual media.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
💬
0
The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling
Hana Pokojn'a, Tobias Isenberg, Stefan Bruckner, Barbora Kozl'ikov'a, Laura Garrison
We apply an approach from cognitive linguistics by mapping Conceptual Metaphor Theory (CMT) to the visualization domain to address patterns of visual conceptual metaphors that are often used in science infographics. Metaphors play an essential part in visual communication and are frequently employed to explain complex concepts. However, their use is often based on intuition, rather than following a formal process. At present, we lack tools and language for understanding and describing metaphor use in visualization to the extent where taxonomy and grammar could guide the creation of visual components, e.g., infographics. Our classification of the visual conceptual mappings within scientific representations is based on the breakdown of visual components in existing scientific infographics. We demonstrate the development of this mapping through a detailed analysis of data collected from four domains (biomedicine, climate, space, and anthropology) that represent a diverse range of visual conceptual metaphors used in the visual communication of science. This work allows us to identify patterns of visual conceptual metaphor use within the domains, resolve ambiguities about why specific conceptual metaphors are used, and develop a better overall understanding of visual metaphor use in scientific infographics. Our analysis shows that ontological and orientational conceptual metaphors are the most widely applied to translate complex scientific concepts. To support our findings we developed a visual exploratory tool based on the collected database that places the individual infographics on a spatio-temporal scale and illustrates the breakdown of visual conceptual metaphors.
Read more7/19/2024
0
Seeing the Unseen: Visual Metaphor Captioning for Videos
Abisek Rajakumar Kalarani, Pushpak Bhattacharyya, Sumit Shekhar
Metaphors are a common communication tool used in our day-to-day life. The detection and generation of metaphors in textual form have been studied extensively but metaphors in other forms have been under-explored. Recent studies have shown that Vision-Language (VL) models cannot understand visual metaphors in memes and adverts. As of now, no probing studies have been done that involve complex language phenomena like metaphors with videos. Hence, we introduce a new VL task of describing the metaphors present in the videos in our work. To facilitate this novel task, we construct and release a manually created dataset with 705 videos and 2115 human-written captions, along with a new metric called Average Concept Distance (ACD), to automatically evaluate the creativity of the metaphors generated. We also propose a novel low-resource video metaphor captioning system: GIT-LLaVA, which obtains comparable performance to SoTA video language models on the proposed task. We perform a comprehensive analysis of existing video language models on this task and publish our dataset, models, and benchmark results to enable further research.
Read more6/10/2024
0
A framework for annotating and modelling intentions behind metaphor use
Gianluca Michelli, Xiaoyu Tong, Ekaterina Shutova
Metaphors are part of everyday language and shape the way in which we conceptualize the world. Moreover, they play a multifaceted role in communication, making their understanding and generation a challenging task for language models (LMs). While there has been extensive work in the literature linking metaphor to the fulfilment of individual intentions, no comprehensive taxonomy of such intentions, suitable for natural language processing (NLP) applications, is available to present day. In this paper, we propose a novel taxonomy of intentions commonly attributed to metaphor, which comprises 9 categories. We also release the first dataset annotated for intentions behind metaphor use. Finally, we use this dataset to test the capability of large language models (LLMs) in inferring the intentions behind metaphor use, in zero- and in-context few-shot settings. Our experiments show that this is still a challenge for LLMs.
Read more7/8/2024
🤔
0
Probing Conceptual Understanding of Large Visual-Language Models
Madeline Schiappa, Raiyaan Abdullah, Shehreen Azad, Jared Claypoole, Michael Cogswell, Ajay Divakaran, Yogesh Rawat
In recent years large visual-language (V+L) models have achieved great success in various downstream tasks. However, it is not well studied whether these models have a conceptual grasp of the visual content. In this work we focus on conceptual understanding of these large V+L models. To facilitate this study, we propose novel benchmarking datasets for probing three different aspects of content understanding, 1) textit{relations}, 2) textit{composition}, and 3) textit{context}. Our probes are grounded in cognitive science and help determine if a V+L model can, for example, determine if snow garnished with a man is implausible, or if it can identify beach furniture by knowing it is located on a beach. We experimented with many recent state-of-the-art V+L models and observe that these models mostly textit{fail to demonstrate} a conceptual understanding. This study reveals several interesting insights such as that textit{cross-attention} helps learning conceptual understanding, and that CNNs are better with textit{texture and patterns}, while Transformers are better at textit{color and shape}. We further utilize some of these insights and investigate a textit{simple finetuning technique} that rewards the three conceptual understanding measures with promising initial results. The proposed benchmarks will drive the community to delve deeper into conceptual understanding and foster advancements in the capabilities of large V+L models. The code and dataset is available at: url{https://tinyurl.com/vlm-robustness}
Read more4/29/2024