So, this is Google Translate built running locally in Chrome? I wonder if it is a small/degraded model or limited languages? Otherwise, how is it not a simple way around the paid Google API?
The article explains that this feature uses a small (up to 22 GB) language model that runs on-device.
That said, the "simple way around the paid API" problem is something Google has to deal with anyway, because there are a bunch of ways to use Google Translate without paying for it (e.g., the translate.google.com web UI, or the non-JavaScript-exposed "Translate to [language]" feature built into Chrome), and any action that can be taken by a human can in principle also be taken by a script. The only thing to do about it is use secret-sauce server-side abuse detection to block as much of this activity as they can; they can't get all of it but they can get enough to push enough people onto the paid API that the economics pencil out.
This sounds off by an order of magnitude? Firefox's local translation models are only 20-70MB per language pair direction (e.g. en-to-fr or fr-to-en).[1] These models are also only released when they reach at least -5% of Google Translate's COMET score.[1] Currently Firefox ships with support for 32 xx-to-en language pairs and 29 en-to-xx language pairs.[1] As the number of language pairs increases, it probably isn't unreasonable for browsers to stop bundling every language pair and instead prompt users to download uncommon models the first time the user wants to use them.
Does the API trigger the download automatically, or does it ask for user permission?
(Answered my own question): Doesn't look like it requires the user's permission. Upon first use, the model will start downloading. The user has to wait for the download to finish before the API will work. That could take hours for 22gb.
Seems very backwards for markets where wired/wifi connections at home are nonexistant, 4g/5g is already unmetered and phones are the wifi you connect your devices to.
It runs very slow. Test case that run translation for text in 3k chars multiple times, takes about 30 seconds for google translator, but more than 10 minutes for `nllb-200-distilled-600M`.
That said, the "simple way around the paid API" problem is something Google has to deal with anyway, because there are a bunch of ways to use Google Translate without paying for it (e.g., the translate.google.com web UI, or the non-JavaScript-exposed "Translate to [language]" feature built into Chrome), and any action that can be taken by a human can in principle also be taken by a script. The only thing to do about it is use secret-sauce server-side abuse detection to block as much of this activity as they can; they can't get all of it but they can get enough to push enough people onto the paid API that the economics pencil out.