Hacker News new | ask | show | jobs
by ricketycricket 745 days ago
> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.

May I ask to which service you are referring?

1 comments

This one: https://jonatanklosko-bumblebee-tools.hf.space/apps/tokenize...

It's linked in the Bumblebee README. Seems broken at the moment, maybe the PR it made is more informative: https://huggingface.co/Neprox/STT-Swedish-Whisper/discussion...

The tokenizer generator has risen again.