free-vc
Maintainer: jagilley
65
| Property | Value |
|---|---|
| Run this model | Run on Replicate |
| API spec | View on Replicate |
| Github link | View on Github |
| Paper link | View on Arxiv |
Create account to get full access
Model overview
The free-vc model is a tool developed by jagilley that allows you to change the voice of spoken text. It can be used to convert the audio of one person's voice to sound like another person's voice. This can be useful for applications like voice over, dubbing, or text-to-speech. The free-vc model is similar in capabilities to other voice conversion models like VoiceConversionWebUI, incredibly-fast-whisper, voicecraft, and styletts2.
Model inputs and outputs
The free-vc model takes two inputs: a source audio file containing the words that should be spoken, and a reference audio file containing the voice that the resulting audio should have. The model then outputs a new audio file with the source text spoken in the voice of the reference audio.
Inputs
- Source Audio: The audio file containing the words that should be spoken
- Reference Audio: The audio file containing the voice that the resulting audio should have
Outputs
- Output Audio: The new audio file with the source text spoken in the voice of the reference audio
Capabilities
The free-vc model can be used to change the voice of any spoken audio, allowing you to convert one person's voice to sound like another. This can be useful for a variety of applications, such as voice over, dubbing, or text-to-speech.
What can I use it for?
The free-vc model can be used for a variety of applications, such as:
- Voice Over: Convert the voice in a video or audio recording to sound like a different person.
- Dubbing: Change the voice in a foreign language film or video to match the local language.
- Text-to-Speech: Generate audio of text spoken in a specific voice.
Things to try
Some ideas for things to try with the free-vc model include:
- Experiment with different source and reference audio files to see how the resulting audio sounds.
- Try using the model to create a voice over or dub for a short video or audio clip.
- See if you can use the model to generate text-to-speech audio in a specific voice.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Models
rvc-v2
19
The rvc-v2 model is a speech-to-speech tool that allows you to apply voice conversion to any audio input using any RVC v2 trained AI voice model. It is developed and maintained by pseudoram. Similar models include the realistic-voice-cloning model for creating song covers, the create-rvc-dataset model for building your own RVC v2 dataset, the free-vc model for changing voice for spoken text, the vqmivc model for one-shot voice conversion, and the metavoice model for a large-scale base model for voice conversion. Model inputs and outputs The rvc-v2 model takes an audio file as input and allows you to convert the voice to any RVC v2 trained AI voice model. The output is the audio file with the converted voice. Inputs Input Audio**: The audio file to be converted. RVC Model**: The specific RVC v2 trained AI voice model to use for the voice conversion. Pitch Change**: Adjust the pitch of the AI vocals in semitones. F0 Method**: The pitch detection algorithm to use, either 'rmvpe' for clarity or 'mangio-crepe' for smoother vocals. Index Rate**: Control how much of the AI's accent to leave in the vocals. Filter Radius**: Apply median filtering to the harvested pitch results. RMS Mix Rate**: Control how much to use the original vocal's loudness or a fixed loudness. Protect**: Control how much of the original vocals' breath and voiceless consonants to leave in the AI vocals. Output Format**: Choose between WAV for best quality or MP3 for smaller file size. Outputs Converted Audio**: The input audio file with the voice converted to the selected RVC v2 model. Capabilities The rvc-v2 model can effectively change the voice in any audio input to a specific RVC v2 trained AI voice. This can be useful for tasks like creating song covers, changing the voice in videos or recordings, or even generating novel voices for various applications. What can I use it for? The rvc-v2 model can be used for a variety of projects and applications. For example, you could use it to create song covers with a unique AI-generated voice, or to change the voice in videos or audio recordings to a different persona. It could also be used to generate novel voices for audiobooks, video game characters, or other voice-based applications. The model's flexibility and the wide range of available voice models make it a powerful tool for voice conversion and generation tasks. Things to try One interesting thing to try with the rvc-v2 model is to experiment with the different pitch, index rate, and filtering options to find the right balance of clarity, smoothness, and authenticity in the converted voice. You could also try combining the rvc-v2 model with other audio processing tools to create more complex voice transformations, such as adding effects or mixing the converted voice with the original. Additionally, you could explore training your own custom RVC v2 voice models and using them with the rvc-v2 tool to create unique, personalized voice conversions.
Updated Invalid Date
📊
openvoice
9
The openvoice model, developed by the team at MyShell, is a versatile instant voice cloning AI that can accurately clone the tone color and generate speech in multiple languages and accents. It offers flexible control over voice styles, such as emotion and accent, as well as other style parameters like rhythm, pauses, and intonation. The model also supports zero-shot cross-lingual voice cloning, allowing it to generate speech in languages not present in the training dataset. The openvoice model builds upon several excellent open-source projects, including TTS, VITS, and VITS2. It has been powering the instant voice cloning capability of myshell.ai since May 2023 and has been used tens of millions of times by users worldwide, witnessing explosive growth on the platform. Model inputs and outputs Inputs Audio**: The reference audio used to clone the tone color. Text**: The text to be spoken by the cloned voice. Speed**: The speed scale of the output audio. Language**: The language of the audio to be generated. Outputs Output**: The generated audio in the cloned voice. Capabilities The openvoice model excels at accurate tone color cloning, flexible voice style control, and zero-shot cross-lingual voice cloning. It can generate speech in multiple languages and accents, while allowing for granular control over voice styles, including emotion and accent, as well as other parameters like rhythm, pauses, and intonation. What can I use it for? The openvoice model can be used for a variety of applications, such as: Instant voice cloning for audio, video, or gaming content Customized text-to-speech for assistants, chatbots, or audiobooks Multilingual voice acting and dubbing Voice conversion and style transfer Things to try With the openvoice model, you can experiment with different input reference audios to clone a wide range of voices and accents. You can also play with the style parameters to create unique and expressive speech outputs. Additionally, you can explore the model's cross-lingual capabilities by generating speech in languages not present in the training data.
Updated Invalid Date
realistic-voice-cloning
299
The realistic-voice-cloning model, created by zsxkib, is an AI model that can create song covers by cloning a specific voice from audio files. It builds upon the Realistic Voice Cloning (RVC v2) technology, allowing users to generate vocals in the style of any RVC v2 trained voice. This model offers an alternative to similar voice cloning models like create-rvc-dataset, openvoice, free-vc, train-rvc-model, and voicecraft, each with its own unique features and capabilities. Model inputs and outputs The realistic-voice-cloning model takes a variety of inputs that allow users to fine-tune the generated vocals, including the RVC model to use, pitch changes, reverb settings, and more. The output is a generated audio file in either MP3 or WAV format, containing the original song's vocals replaced with the cloned voice. Inputs Song Input**: The audio file to use as the source for the song RVC Model**: The specific RVC v2 model to use for the voice cloning Pitch Change**: Adjust the pitch of the AI-generated vocals Index Rate**: Control the balance between the AI's accent and the original vocals RMS Mix Rate**: Adjust the balance between the original vocal's loudness and a fixed loudness Filter Radius**: Apply median filtering to the harvested pitch results Pitch Detection Algorithm**: Choose between different pitch detection algorithms Protect**: Control the amount of original vocals' breath and voiceless consonants to leave in the AI vocals Reverb Size, Damping, Dryness, and Wetness**: Adjust the reverb settings Pitch Change All**: Change the pitch/key of the background music, backup vocals, and AI vocals Volume Changes**: Adjust the volume of the main AI vocals, backup vocals, and background music Outputs The generated audio file in either MP3 or WAV format, with the original vocals replaced by the cloned voice Capabilities The realistic-voice-cloning model can create high-quality song covers by replacing the original vocals with a cloned voice. Users can fine-tune the generated vocals to achieve their desired sound, adjusting parameters like pitch, reverb, and volume. This model is particularly useful for musicians, content creators, and audio engineers who want to create unique vocal covers or experiments with different voice styles. What can I use it for? The realistic-voice-cloning model can be used to create song covers, remixes, and other audio projects where you want to replace the original vocals with a different voice. This can be useful for musicians who want to experiment with different vocal styles, content creators who want to create unique covers, or audio engineers who need to modify existing vocal tracks. The model's ability to fine-tune the generated vocals also makes it suitable for professional audio production work. Things to try With the realistic-voice-cloning model, you can try creating unique song covers by cloning the voice of your favorite singers or even your own voice. Experiment with different RVC models, pitch changes, and reverb settings to achieve the desired sound. You could also explore using the model to create custom vocal samples or background vocals for your music productions. The versatility of the model allows for a wide range of creative possibilities.
Updated Invalid Date
metavoice
9
MetaVoice-1B is a 1.2 billion parameter base model trained on 100,000 hours of speech, developed by the MetaVoice team. This large-scale speech model can be used for a variety of text-to-speech and audio generation tasks, similar to models like ml-mgie, meta-llama-3-8b-instruct, whisperspeech-small, voicecraft, and whisperx. Model inputs and outputs MetaVoice-1B takes in text as input and generates audio as output. The model can be used for a wide range of text-to-speech and audio generation tasks. Inputs Text**: The text to be converted to speech. Outputs Audio**: The generated audio in a URI format. Capabilities MetaVoice-1B is a powerful foundational audio model capable of generating high-quality speech from text inputs. It can be used for tasks like text-to-speech, audio synthesis, and voice cloning. What can I use it for? The MetaVoice-1B model can be used for a variety of applications, such as creating audiobooks, podcasts, or voice assistants. It can also be used to generate synthetic voices for video games, movies, or other multimedia projects. Additionally, the model can be fine-tuned for specific use cases, such as language learning or accessibility applications. Things to try With MetaVoice-1B, you can experiment with generating speech in different styles, emotions, or languages. You can also explore using the model for tasks like audio editing, voice conversion, or multi-speaker audio generation.
Updated Invalid Date