Creating Aesthetic Sonifications on the Web with SIREN

Introduction

The paper describes the challenge of finding an optimal balance between increasing the utility of a sonification system while keeping it accessible for anyone to use. It discusses how different sonification interfaces have landed at various points along this spectrum, with some being more opinionated in their controls and others aiming for a more universal design.

The Sonification Interface for REmapping Nature (SIREN) attempts to address this tension by providing an accessible user interface and intuitive workflow, despite having a slightly higher threshold in some aspects compared to other sonification interfaces. SIREN's main approach is parameter mapping sonification, which is a powerful and multidimensional method for data sonification.

The paper focuses on the improvements in SIREN's version 0.4 and the range of sound synthesis methods that SIREN supports, which makes it a versatile and general-purpose data sonification tool. Future work will explore expanding SIREN's capabilities with additional sound synthesis methods.

Related Work

The provided text discusses the extensibility and customizability of various sonification interfaces. It summarizes several existing sonification tools and the range of parameters they offer for users to customize their sonifications.

MUsical Sonification Environment focused on musical parameters like instrument timbres, rhythms, tempo, and harmony. MUSART introduced an innovative audio transfer function that allows mapping data to up to 10 auditory parameters. SonART provides flexible parameter mapping controls by enabling users to leverage the synthesis capabilities of the Synthesis ToolKit. Sonification Sandbox includes pitch, timbre, gain, and pan controls, as well as contextual sonic notifications.

The Interactive Sonification Toolkit supports parameter mapping as well as audification. Sonification Integrative Flexible Toolkit offers parameters like pitch, gain, timbre, duration, and pan. xSonify, designed for visually-impaired users, has a more limited set of customizable parameters - pitch, gain, and rhythm.

Sonifyer primarily uses audification, but also supports parameter mapping through frequency modulation synthesis. Rotator incorporates six synthesizer types into its sonification capabilities. Sonification Workstation uses a visual programming approach, allowing users to patch together components with adjustable settings.

WebAudioXML Sonification Toolkit provides an XML-based interface for sonification, with parameters like pitch, brightness, gain, and filter. Highcharts Sonification Studio, a modern web-based successor to Sonification Sandbox, offers parameters such as pitch, pan, gain, timbre, lowpass/highpass filters, and tremolo.

The text highlights the importance of extensibility and customizability in sonification interfaces, as it allows users to tailor the sonifications to their specific needs and preferences.

SIREN

The paper presents SIREN, a web-based interface for creating sonifications. Some key points:

Design: SIREN's interface is inspired by digital audio workstations, allowing users to easily visualize multidimensional and polyphonic sonifications. The main organizational unit is the "track", which contains one parameter-mapped synthesizer and regions that hold data.

Architecture: SIREN uses modern tools like SvelteKit, Vite, and TypeScript.

Workflow: Creating sonifications in SIREN involves 1) importing datasets, 2) adding synthesizers, 3) creating tracks and regions, and 4) applying data mappings.

Sound Synthesis Paradigm: SIREN allows users to create their own synthesizers as JavaScript modules with a defined structure. This includes specifying timbral and temporal parameters, update functions, and optional AudioWorklets.

The paper provides three working examples of synthesizers built for SIREN: Frequency Modulation, Formant, and Granular synthesis. These demonstrate the flexibility and extensibility of SIREN's sound synthesis approach.

Future Work

The paper discusses ways to expand the capabilities of the SIREN sound synthesis interface. It notes that while SIREN can support a wide range of sound synthesis techniques, there is still potential to incorporate more types of synthesis to further demonstrate SIREN's capabilities as a general-purpose data sonification tool.

The paper mentions another sonification interface called SonART, which uses the Synthesis ToolKit (STK) to provide sonification parameters. Future work could involve compiling STK into WebAssembly, allowing SIREN to also utilize STK for parameter mapping sonification. WebAssembly also enables the integration of other sound synthesis technologies like WebChucK, Faust RNBO, and Csound, which could expand the types of synthesizers available for SIREN.

Additionally, the paper suggests creating an audification synthesizer for SIREN, as audification is a common sonification method used by other interfaces. Finally, the paper proposes an open-source synthesizer repository where users can contribute new synthesizer implementations using JavaScript and the Web Audio API, which could help expand SIREN's general-purpose capabilities.

Conclusion

SIREN aims to create a data sonification interface that can produce high-quality audio while remaining accessible. The paper demonstrates several examples of aesthetic sonifications achieved through SIREN's design. Future work will focus on developing additional synthesizers to expand the range of possible synthesized sounds and parameters.