Scrolly2Reel: Retargeting Graphics for Social Media Using Narrative Beats
2403.18111
1
0
Abstract
Content retargeting is crucial for social media creators. Once great content is created, it is important to reach as broad an audience as possible. This is particularly important in journalism where younger audiences are shifting away from print and towards short-video platforms. Many newspapers already create rich graphics for the web that they want to be able to reuse for social media. One example is scrollytelling sequences or scrollies -- immersive articles with graphics like animation, charts, and 3D visualizations that appear as a user scrolls. We present a system that helps transform scrollies into social media videos. By using the scriptwriting concept of narrative beats to extract fundamental storytelling units, we can create videos that are more aligned with narration, and allow for better pacing and stylistic changes. Narrative beats are thus an important primitive to retargeting content that matches the style of a new medium while maintaining the cohesiveness of the original content.
Create account to get full access
Overview
ā¢ This paper introduces a system called Scrolly2Reel that can transform news graphics into short-form video content for social media platforms like TikTok.
ā¢ The key innovations are techniques to adjust the narrative pacing and beats of the content to better match the expectations and conventions of social media video formats.
ā¢ The authors demonstrate how their approach can be used to repurpose and retarget existing news graphics content for more engaging social media experiences.
Plain English Explanation
The researchers have developed a system called Scrolly2Reel that can take news graphics, like the type you might see in a news article or on a website, and turn them into short video clips suitable for platforms like TikTok. The main challenge they wanted to address is that the pacing and structure of traditional news graphics don't always work well when viewed as a quick social media video.
So Scrolly2Reel uses some clever techniques to adjust the "narrative beats" and overall pacing of the content. This helps make the information more engaging and digestible in a short video format. The authors show how their system can take existing news graphics and repurpose them to work better on social media, without having to create brand new content from scratch.
This is an interesting approach because it allows news organizations and other content creators to extend the life and reach of their existing graphics by optimizing them for platforms like TikTok, where short-form video is very popular. It's a way to repurpose and retarget content to new formats and audiences, without having to start over.
Technical Explanation
The Scrolly2Reel system takes news graphics as input and applies several key techniques to transform them into short-form video content:
-
Narrative Beat Alignment: The system analyzes the narrative structure of the news graphic and identifies key "beats" or moments that drive the story forward. It then adjusts the pacing and timing of these beats to better match the expected cadence of social media video formats.
-
Pacing Adjustment: In addition to beat alignment, Scrolly2Reel also adjusts the overall pacing of the content, speeding up or slowing down different sections to create a more engaging, TikTok-friendly rhythm.
-
GPT-Shortening: The system uses large language models like GPT to generate concise, punchy captions and text overlays that convey the key information in a more compact way suitable for short videos.
-
Repurposing and Retargeting: By applying these techniques, Scrolly2Reel can take existing news graphics and repurpose them into short-form video content targeted specifically for social media platforms and audiences.
The authors evaluate their system through both quantitative and qualitative studies, demonstrating its ability to create engaging TikTok-style videos from traditional news graphics while preserving the core informational content.
Critical Analysis
The Scrolly2Reel system presents an interesting approach to repurposing news graphics for social media platforms, but there are a few potential limitations and areas for further research:
-
The system relies heavily on the quality and accuracy of the underlying news graphics - if the original content is unclear or misleading, the short-form video may inherit those issues.
-
While the pacing and beat alignment techniques are novel, their effectiveness likely depends on a deep understanding of social media video conventions, which could vary across platforms and user demographics.
-
The use of large language models for text generation introduces potential risks around biases, factual accuracy, and coherence that would need to be carefully monitored.
Further research could explore ways to incorporate user feedback and engagement data to dynamically optimize the Scrolly2Reel content, as well as investigations into the long-term impact of this type of repurposed news content on social media platforms.
Conclusion
Overall, the Scrolly2Reel system represents a promising approach to bridging the gap between traditional news graphics and the short-form video formats preferred on social media. By applying techniques to adjust narrative pacing and structure, the system can breathe new life into existing news content and make it more engaging and accessible to younger, social media-savvy audiences. As news organizations and content creators continue to grapple with the challenges of reaching users on platforms like TikTok, tools like Scrolly2Reel may become increasingly valuable for repurposing and retargeting their valuable informational assets.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
šæ
Synchronized Video Storytelling: Generating Video Narrations with Structured Storyline
Dingyi Yang, Chunru Zhan, Ziheng Wang, Biao Wang, Tiezheng Ge, Bo Zheng, Qin Jin
0
0
Video storytelling is engaging multimedia content that utilizes video and its accompanying narration to attract the audience, where a key challenge is creating narrations for recorded visual scenes. Previous studies on dense video captioning and video story generation have made some progress. However, in practical applications, we typically require synchronized narrations for ongoing visual scenes. In this work, we introduce a new task of Synchronized Video Storytelling, which aims to generate synchronous and informative narrations for videos. These narrations, associated with each video clip, should relate to the visual content, integrate relevant knowledge, and have an appropriate word count corresponding to the clip's duration. Specifically, a structured storyline is beneficial to guide the generation process, ensuring coherence and integrity. To support the exploration of this task, we introduce a new benchmark dataset E-SyncVidStory with rich annotations. Since existing Multimodal LLMs are not effective in addressing this task in one-shot or few-shot settings, we propose a framework named VideoNarrator that can generate a storyline for input videos and simultaneously generate narrations with the guidance of the generated or predefined storyline. We further introduce a set of evaluation metrics to thoroughly assess the generation. Both automatic and human evaluations validate the effectiveness of our approach. Our dataset, codes, and evaluations will be released.
5/24/2024
š¤Æ
RetAssist: Facilitating Vocabulary Learners with Generative Images in Story Retelling Practices
Qiaoyi Chen, Siyu Liu, Kaihui Huang, Xingbo Wang, Xiaojuan Ma, Junkai Zhu, Zhenhui Peng
0
0
Reading and repeatedly retelling a short story is a common and effective approach to learning the meanings and usages of target words. However, learners often struggle with comprehending, recalling, and retelling the story contexts of these target words. Inspired by the Cognitive Theory of Multimedia Learning, we propose a computational workflow to generate relevant images paired with stories. Based on the workflow, we work with learners and teachers to iteratively design an interactive vocabulary learning system named RetAssist. It can generate sentence-level images of a story to facilitate the understanding and recall of the target words in the story retelling practices. Our within-subjects study (N=24) shows that compared to a baseline system without generative images, RetAssist significantly improves learners' fluency in expressing with target words. Participants also feel that RetAssist eases their learning workload and is more useful. We discuss insights into leveraging text-to-image generative models to support learning tasks.
5/24/2024
š¤
ID.8: Co-Creating Visual Stories with Generative AI
Victor Nikhil Antony, Chien-Ming Huang
0
0
Storytelling is an integral part of human culture and significantly impacts cognitive and socio-emotional development and connection. Despite the importance of interactive visual storytelling, the process of creating such content requires specialized skills and is labor-intensive. This paper introduces ID.8, an open-source system designed for the co-creation of visual stories with generative AI. We focus on enabling an inclusive storytelling experience by simplifying the content creation process and allowing for customization. Our user evaluation confirms a generally positive user experience in domains such as enjoyment and exploration, while highlighting areas for improvement, particularly in immersiveness, alignment, and partnership between the user and the AI system. Overall, our findings indicate promising possibilities for empowering people to create visual stories with generative AI. This work contributes a novel content authoring system, ID.8, and insights into the challenges and potential of using generative AI for multimedia content creation.
6/4/2024
Story Generation from Visual Inputs: Techniques, Related Tasks, and Challenges
Daniel A. P. Oliveira, Eug'enio Ribeiro, David Martins de Matos
0
0
Creating engaging narratives from visual data is crucial for automated digital media consumption, assistive technologies, and interactive entertainment. This survey covers methodologies used in the generation of these narratives, focusing on their principles, strengths, and limitations. The survey also covers tasks related to automatic story generation, such as image and video captioning, and visual question answering, as well as story generation without visual inputs. These tasks share common challenges with visual story generation and have served as inspiration for the techniques used in the field. We analyze the main datasets and evaluation metrics, providing a critical perspective on their limitations.
6/6/2024