Whisper tokenizer is working something wrong

[mrmuminov]

I use whiper-small tokenizer of Uzbek language.
It's working but, something wrong.
Because, it's returned non-Uzbek characters.
How good (correct) it is?

How to fix that?
Can I make custom tokenizer? (share some articles).

[phineas-pta]

whisper accuracy on uzbek language is very bad (WER > 90% see paper), so u may want to fine tune

custom tokenizer meaning u have to re-train whisper from scratch

[mrmuminov]

@phineas-pta

I do fine-tuning through the medum model.
But still WER is not what I expected. It is doing worse.
I think, that this poor result is due to the "wrong working" in WhisperTokenizer.

For example, in the image above, there are excess characters (marked with a red line).

So I was wondering if I could rewrite WhisperTokenizer. Is it really the right thing to do? I don't know if it can be done.

[phineas-pta]

fine tuning give worse WER ? 🤔 make sure your data is properly cleaned + try train longer

u can ofc create your own tokenizer but as i said it'll require re-train

[mrmuminov]

@phineas-pta

u can ofc create your own tokenizer but as i said it'll require re-train

I think I understand what you mean, I will try it and let you know the result.