fine tuning whisper locally

[Atefeh197]

Hello every one

I need to fine-tune Whisper but locally on my private data, I saw some useful links to fine-tune Whisper on hugging face, but I need to do that locally, Is there any way?

Thanks

[itaipee]

most of guide are some variation of the original fine-tune guide from open-AI https://huggingface.co/blog/fine-tune-whisper

some of the pre-processing stages need to be different , since you use your own data and not a prepared data in HF,
you need to make sure you set the 'saving' arguments in Seq2SeqTrainingArguments , output_dir , and save_steps , and also set push_to_hub=False. Also ,and skip the command trainer.push_to_hub(**kwargs)

afterward , the model is saved locally , under <output_dir>/checkpoint-

for transcribing (locally) later with the fine-tuned model , you need to do some conversion from the fine-tuned locall HF model , to 'regular' whisper model , good reference for this here : https://medium.com/axinc-ai/whisper-fine-tuning-to-transcribe-jargon-976164a5eac8