Ultimate Vocal Remover API v0.1

This is a an API for ultimate vocal removing. It is designed to be expandable with new models/algorethems while maintaining a simple interface. Colab demo

Install

If you intend to edit the code

git clone https://github.com/NextAudioGen/ultimatevocalremover_api.git
cd ultimatevocalremover_api
pip install .

Usage

import uvr
from uvr import models
from uvr.utils.get_models import download_all_models
import torch
import audiofile
import json

models_json = json.load(open("/content/ultimatevocalremover_api/src/models_dir/models.json", "r"))
download_all_models(models_json)
name = {name_of_your_audio}
device = "cuda"
    
demucs = models.Demucs(name="hdemucs_mmi", other_metadata={"segment":2, "split":True}, device=device, logger=None)

# Separating an audio file
res = demucs(name)
seperted_audio = res["separated"]
vocals = seperted_audio["vocals"]
base = seperted_audio["bass"]
drums = seperted_audio["drums"]
other = seperted_audio["other"]

Archetecture:

Ultimate Vocal Remover API
├── src
│   ├── audiotools.py 
│   ├── models.py 
│   ├── ensembles.py
│   ├── pipelines.py
│   ├── utils/
│   ├── audio_tools/
│   └── models_dir
│       ├── Each implementation of a model is added here as a single directory.
│       └── models.json (this is used to download the models)
├── docs
│   ├── models/
│   │   └── Here goes all models docs each in a single directory.
│   ├── ensembles/
│   │   └── Here goes all ensembles docs each in a single directory.
│   ├── pipelines/
│   │   └── Here goes all pipelines docs each in a single directory.
│   ├── audio_tools/
│   └── utils/
└── tests/
    ├── test_models.py
    ├── test_ensembles.py
    ├── test_pipelines.py
    ├── test_audiotools.py
    └── utils/

audiotools.py: Interface for all audio tools
models.py: Interface for all models following a consistent interface
utils/ Here goes read and write utils for audio, models...etc. \

All models, pipelines and ensembles follow this interface:

class BaseModel:
    def __init__(self, name:str, architecture:str, other_metadata:dict, device=None, logger=None)
    def __call__(self, audio:Union[npt.NDArray, str], sampling_rate:int=None, **kwargs)->dict
    # @singledispatch
    def predict(self, audio:npt.NDArray, sampling_rate:int, **kwargs)->dict
    def predict_path(self, audio:str, **kwargs)->dict
    def separate(self, audio:npt.NDArray, sampling_rate:int=None)->dict
    def __repr__(self)
    def to(self, device:str)
    def update_metadata(self, metadata:dict)
    @staticmethod
    def list_models()->list

Contribution

If you like this, leave a star, fork it, and definitely you are welcomed to buy me a coffee.

Also, please open issues, make pull requests but remember to follow the structure and interfaces. Moreover, we are trying to build automated testing, we are aware that the current tests are so naive but we are working on it. So please make sure to add some tests to your new code as well.

Refrences

code

Code and weights from these sources used in developing this library:

• MDX-Net This is the original MDX architecture implementation.
• MDXC and demucs This repo has a clever ensumbling methods for MDX, Demucs 3, and Demucs 4. Moreover they have the wieghts for their finetuned MDX open (available under MDXC implementation here).
• Demucs This is the original implementation of the model.
• ultimatevocalremovergui This is one of the best vocal removers. A lot of ideas in this repo were borrowed from here.
• weights Most of the models right now are comming from this repo.

Papers

• Benchmarks and leaderboards for sound demixing tasks
• MULTI-SCALE MULTI-BAND DENSENETS FOR AUDIO SOURCE SEPARATION
• HYBRID TRANSFORMERS FOR MUSIC SOURCE SEPARATION
• KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixing

Core Developers

• Mohannad Barakat
• Noha Magdy
• Mohtady Ehab