[tbenst]

This is a super cool device. Note that the decoding is highly limited: they decode into one of five different sentences. This is easier than five words for example as there is more information to distinguish.

Unfortunately the media is blowing this way of out proportion as the larynx alone does not contain sufficient information to decode silent speech.

If you also sense the lips, tongue articulators, and jaw, then general English decoding becomes possible with high accuracy (eg see our recent work here: https://x.com/tbenst/status/1767952614157848859). It’s not in the preprint but I’ve done experiments with only the larynx recorded and performance is pretty abysmal on even a 10 word vocabulary—-hence why they did a five sentence task.

[ImHereToVote]

I bet if you listened to the feedback you could teach yourself to talk using the larynx and surrounding muscles.

[jvanderbot]

Why can't the muscles of the larnex and perhaps chest / diaphragm, be monitored and mapped to vocal chord noises, rather than full speech? Just put the noise in the throat and let the rest of the body make it work.

[irviss]

> If you also sense the lips, tongue articulators, and jaw, then general English decoding becomes possible with high accuracy

A bit OT but I see this frequently and I'm curious. Why do you English speakers (or just a US phenomenon?) tend to use the word "English" instead of "language", "linguistic" or one of its related words to refer to a general concept?

[x1798DE]

Not OP, but as a native English speaker and former scientist (though not in this area), I would interpret "x does y on English tasks" to mean "we tested this in English and don't know if the effect generalizes to other languages".

[thaumasiotes]

In this case we do know if the effect generalizes to other languages. It cannot fail to; the larynx, lips, tongue, and jaw are almost all there is. For example, vowels are conventionally defined by jaw position ("height"), tongue position ("frontness"), and lip configuration ("rounded" or not).

You might miss some things like creaky voice or ejectives, you'll probably miss aspiration, but all that does is give you a worst-case scenario analogous to a native speaker trying to understand someone with a foreign accent. Extremely high accuracy will be possible.

[AlecSchueler]

This is a reasonable hypothesis but if only English has been studied then it would be unscientific to extrapolate at this time.

[thaumasiotes]

Sure, in the same sense that it would be "unscientific" to conclude that someone's amputated leg didn't regenerate by chance, because the sample size is only 1.

If you know how you're recognizing English, and you know that other languages do not differ from English in relevant ways, then you know you can recognize those other languages. Pretending you don't know something you do know is not scientific.

[brookst]

This seems like damned-either-way. If they had only tested English and asserted that it was universally applicable to all languages, it’s likely you (or someone else) would rightfully object that it’s annoying when English speakers assume that’s all there is.

[AlecSchueler]

You don't know, though. You have a good working hypothesis and you can make reasoned predictions, but it remains untested. The core principle of science is that we test our hypotheses.

[metabagel]

Other languages have different sounds which aren’t present in English.

[tbenst]

x1798DE captured my intent well. For example, tonal languages like Mandarin or Cantonese may be more difficult to decode if vocal cords aren’t vibrating, and languages with more phonemes that have both a voiced and unvoiced version might be more difficult. I still think decoding will be possible for general language, but that’s a hypothesis whereas I know it’s true for English.

[thaumasiotes]

> and languages with more phonemes that have both a voiced and unvoiced version might be more difficult.

I had the understanding that English is unusually rich in phonemes that occur in both a voiced and unvoiced version. But as I've mentioned sidethread, this just isn't very significant as far as transcribing English goes.

English has an almost full series of stop and fricative phonemes that exhibit voicing contrasts:

- Bilabial, alveolar, and velar stops /p, b, t, d, k, g/, though the distinction between /t/ and /d/ disappears intervocalically in American English. [In practice, English speakers differentiate these phonemes more by the contrast of aspiration than by the contrast of voicing.]

- Interdental, labiodental, alveolar, palatal, but generally not velar, fricatives /θ, ð, f, v, s, z, ʃ, ʒ/, along with palatal affricates /tʃ, dʒ/.

- Nasals and approximants are always voiced.

Compare a language like Mandarin Chinese, where there are between zero and one pairs of phonemes that contrast by voicing (the sound represented by pinyin "r" may be a voiced fricative otherwise equivalent to "sh", or it may be an approximant; there is no contrasting voiceless approximant), or Spanish, where only the stops feature this contrast.

What are the languages that have more voicing contrasts than English does? It would almost be necessary for such a language to distinguish between voiced and unvoiced vowels. (Some quick research suggests that Icelandic at least has a comparable number of voicing contrasts, but it is not obviously more than English and appears to be actively shrinking.)

> tonal languages like Mandarin or Cantonese may be more difficult to decode if vocal cords aren’t vibrating

More difficult, yes, but in the sense that decoding may take more computation, not that the error rate will go up.

Again, we can already observe that e.g. Mandarin speakers do not have trouble understanding text that carries no information about tone, nor do they have trouble understanding songs, where lexical tone is overridden by the melody of the song.

(What happens here depends what you mean. If you want to decode speech into pinyin with tone marks omitted, the lack of ability to measure tones will fail to be a problem by definition. If you want to decode into Chinese characters, you'll need a robust model of the language, at which point lack of tones will also fail to be a problem - the language model will cover for it. If you want to decode into pinyin with tone marks, you won't be able to do that without using a language model.)

[roenxi]

I'd speculate English speakers are used to being part of a society where non-English speakers are present and politically important. It is polite not to assume that English = language. Even on the British Isles English isn't a universal thing. Let alone somewhere like America where it isn't even native.

"Language" just doesn't mean "English". In Australia if someone is talking about "language" on its own I'd assume they're Aboriginal advocates.

[AlecSchueler]

> Even on the British Isles English isn't a universal thing. Let alone somewhere like America where it isn't even native.

English isn't native to all of those isles either only Great Britain.

[khazhoux]

This is your misperception.

In the instances where a person says "English" in this kind of context, it catches your attention and you infer that the person is an English-speaker, and possibly American.

But when a person uses the generic word "language", you don't notice it.

This leads you to believe that English speakers "tend to use the word English," when that's not the case necessarily.

I don't know what this perceptual fallacy is called, but there's probably a word. In English :-)

[atopal]

There are about 6000 spoken languages around the world with an extreme variety in how they produce meaning. How could you make sweeping statements about all of them?

[johnisgood]

I have not noticed this. I just assume that they are specifically talking about a language, in this case: English.