Music-and-culture-technology-lab

omnizart

music-and-culture-technology-lab

3

Omnizart is a Python library developed by the Music and Culture Technology (MCT) Lab that aims to democratize automatic music transcription. It can transcribe various musical elements such as pitched instruments, vocal melody, chords, drum events, and beat from polyphonic audio. Omnizart is powered by research outcomes from the MCT Lab and has been published in the Journal of Open Source Software (JOSS). Similar AI models in this domain include music-classifiers for music classification, piano-transcription for high-resolution piano transcription, mustango for controllable text-to-music generation, and musicgen for music generation from prompts or melodies. Model inputs and outputs Omnizart takes an audio file in MP3 or WAV format as input and can output transcriptions for various musical elements. Inputs audio**: Path to the input music file in MP3 or WAV format. mode**: The specific transcription task to perform, such as music-piano, chord, drum, vocal, vocal-contour, or beat. Outputs The output is an array of objects, where each object contains: file: The path to the input audio file. text: The transcription result as text. Capabilities Omnizart can transcribe a wide range of musical elements, including pitched instruments, vocal melody, chords, drum events, and beat. This allows users to extract structured musical information from audio recordings, enabling applications such as music analysis, music information retrieval, and computer-assisted music composition. What can I use it for? With Omnizart, you can transcribe your favorite songs and explore the underlying musical structure. The transcriptions can be used for various purposes, such as: Music analysis**: Analyze the harmonic progressions, rhythmic patterns, and melodic lines of a piece of music. Music information retrieval**: Extract relevant metadata from audio recordings, such as chord changes, drum patterns, and melody, to enable more sophisticated music search and recommendations. Computer-assisted music composition**: Use the transcribed musical elements as a starting point for creating new compositions or arrangements. Things to try Try using Omnizart to transcribe different genres of music and explore the nuances in how it handles various musical elements. You can also experiment with the different transcription modes to see how the results vary and gain insights into the strengths and limitations of the model.

Read more