[jiehong]

For those not trying, this allows Deepseek to understand a picture (instead of just extracting text from it), and it can describe what's in the picture, but this is not an image generation system, so you can't ask it to modify an image.

Personally, I'm a bit surprised the DS chat app still doesn't offer its own text to speech and speech to text features (I know DS doesn't have any ASR model for example, but there are quite a few in the open).

[testbjjl]

DeepSeek interpreting screenshots and images I send it at fractions of what I pay Claude and ChatGPT, for me, is of far higher priority than supporting dictation. There are workarounds for dictation but not image processing.

[anthonypasq]

just use one of the various cheap gemini models

[freedomben]

Indeed, Gemini really is incredible at image analysis. Yesterday I pointed it at some sloppy handwritten notes and asked it to add up the numbers in the right column, and it did it no problem. I've also used it to find out what TV show or actor is on screen, and various other things. It's quite impressive.

[johnvanommen]

> Indeed, Gemini really is incredible at image analysis. Yesterday I pointed it at some sloppy handwritten notes and asked it to add up the numbers in the right column, and it did it no problem. I've also used it to find out what TV show or actor is on screen, and various other things. It's quite impressive.

I do not know if it works as well as Gemini, but Salesforce (of all places) has a model that does something similar.

What's "neat" about the Salesforce one is that you can run it locally and just iterate it over as many images as you feel like.

For instance, it should be possible to take a movie, pull a hundred images out of the h265 file, have the salesforce model evaluate what is happening at that moment in the movie, and then use that to create an index.

That's just ONE use for it, and I can think of dozens.

On a 5090 it was able to generate text descriptions of a folder full of approximately 500 images in under a minute. (Anecdotal evidence, admittedly.)

https://huggingface.co/Salesforce/blip-image-captioning-base

I just looked up some articles on it here, and it looks like it's fairly old, so YMMV.

[brianjking]

There is a newer BLIP-2, but it's also fairly old. You're better off with many other local models such as Moondream 3 https://huggingface.co/moondream/moondream3-preview.

Moondream is great as it can point, count, perform bounding boxes, descriptions, and visual grounded reasoning.

[winstonp]

Gemini pretty clearly has the best underlying model, and the worst RL and post-training of the lot.

[MattSayar]

I got a shirt I liked from a conference, and I didn't know who made it. It was soft, fit comfortably... I took a picture of some random numbers on a tag and Gemini parsed out the numbers and found the manufacturer. Pretty neat

[carterschonwald]

gemini models are also fantastic at understanding non spoken sounds

[jauntywundrkind]

I don't know what runs on my phone's Google Translate app, but whatever it is, they are doing an insult to their models by it being so bad. It's amazing at picking up sound if spoken directly into the unit, but if trying to hold any kind of conversation or listen to anything even a little bit far away, it falls completely apart, is good for basically nothing.

This is obviously different than the models most people are discussing here, which are much bigger. But it's damaging the Gemini brand in general, by association, if nothing else.

[Royce-CMR]

I’ve long wondered if this was deliberate - only conversations where the participants are overtly using the translator get parsed.

[segmondy]

You can do that with smaller models at home. Gemma-4-E4B will run on a 12gb GPU, and supports audio, image, video input

[NooneAtAll3]

12GB GPU is a lot

[corimaith]

Or you could just use a CNN...

[bigmadshoe]

CNNs are not SoTA anymore when it comes to large models, and also are not used to provide interpretations of images as text, but rather to classify, do semantic segmentation, etc.

[bonoboTP]

CNNs are fine when trained with a good recipe. There are very few good studies comparing them with proper hyperparam search and all the training tricks applied consistently. Transformers are good but ViT vs CNN is not some settled issue. Transformers are more hyped and more popular with the tech enthusiasts who just read forums and news, but if you need stuff done, CNNs are still great.

[bigmadshoe]

I agree, but since we're talking about imagine understanding with text output, clearly a CNN is unsuitable. My previous comment was overly reductive and CNNs can still be SoTA depending on your performance metrics. I spent the earlier part of my career training CNNs, and they are very pleasant to work with.

[bonoboTP]

You can run a CNN and use the downsampled feature map the same way as patch tokens.

[famouswaffles]

>Transformers are more hyped and more popular with the tech enthusiasts who just read forums and news, but if you need stuff done, CNNs are still great.

Vits are straight up more popular for ML research now, it's not just 'tech enthusiasts'.

[bonoboTP]

There's a dearth of research properly comparing them.

[tehjoker]

Can you say more about that? I haven't kept up.

[crypto420]

CNNs excel in vision tasks where you have limited compute, limited memory, limited data, and want something that works super well and quick. People usually don't hook CNNs up to a transformer to get language understanding either, you have to train bespoke CNNs for specific tasks

ViTs excel where you're unbounded in compute + data and also want text understanding or have a conversation about an image

[bonoboTP]

These are vibes. ViT has been shown to work fine on small data with proper hyperparam and most of what you mention is actually doable just fine with the other architecture as well.

[Jabrov]

Transformers are superior

[nullstyle]

Which?

[paulluuk]

Can you explain what the benefits are of actually "talking" with the bot instead of typing and reading?

As someone who would rather send a slack message to a coworker rather than actually walking over and talk to them, the idea of having to talk with my laptop is not appealing at all, haha.

[cicko]

If you spend your life sitting in a chair, that's fine. I tend to get all kinds of ideas, questions, and research needs while I'm walking around. Typing a paragraph or two or context takes too much time and is very risky. Especially when driving. But also just walking, cooking, cleaning, etc. Sometimes it's just not practical - winter, carrying stuff... I mostly feel privileged if I can just sit at a computer and type my question and have the time to read the answer.

[itake]

I am someone that prefers a slack message to a coworker than talking to them and I use AI.

My current flow is: Google Eloquent to capture 127WPM (my typing is best case is 65wpm). This lets me get the thoughts out without thinking too much about structure or flow, the same way I would brain-dump type it.

Next I use AI to compress, summarize, and restructure to create a clear coherent message for my peer to read (which is way faster for them).

When communicating with AI, its the same thing, except I skip the second step since AI does a good job at understanding my ramblings.

----

It drives me crazy that some cultures only send voice messages to each other. It drives me crazy they can't be respectful of my time and use STT+AI to convert their 90 second monologue to a few written sentences.

[garblegarble]

Slightly off-topic but: does it concern you that you're letting atrophy a very important skill for human communication (organising your thoughts and ideas, and then clearly communicating them to others)?

[itake]

Tbh, I never have been a good writer. A college professor once told me I am a terrible writer. I've tried to get better (I read a lot, I write a lot, I've taken multiple college level writing course). I even started a blog (https://kcoleman.me).

I kinda view myself as a wheelchair user. I'm bad at walking so I use at wheelchair so I can at least have a semblance of decent communication. I don't think my ideas are not worth sharing, but I'm just bad at writing them in an engaging way.

The scarier thing for me is coding. I am good at coding. But I don't even read a single line of code any more.

[limflick]

As someone who's still learning English, this is one thing I'd never use AI for, at least not in the near future, simply because thinking and structuring my thoughts before typing is the same as it is before speaking and actually talking to other people can't be outsourced to AI.

But I imagine if I'd been a native speaker I wouldn't mind using AI like OC does since it's a convenience. Same way I use a calculator for two digit multiplications in real life but spent years learning to do it manually in school.

[itake]

You're probably further into english than I am into vietnamese, but I really like using AI to help me improve my vocabulary and understanding of the language.

I avoid using AI as a direct translation tool, but its super useful for me to translate complex english ideas to vietnamese.

[billnad]

As a native English Speaker I can tell you that I would have some trouble talking out an email. I like the back and forth in my head of editing as I go. Text messaging may be fine but email is more difficult for me to just talk through.

I am loving the conversation here though of how people are using speech to talk to LLMs or not though, it is something that no one talks about much

[a34729t]

This worries me tremendously. In fact, it is one of the major points of value that i deliver as an engineer. Organizing and iteration on thoughts is not trivial or easy, but it is very important!

[johnvanommen]

> Organizing and iteration on thoughts is not trivial or easy, but it is very important!

Two of the silliest things that helped me in my career:

* I worked at fast food restaurants in high school. This instills a near pavlovian response to client requests; if at the age of sixteen you can deal with someone who's mad because there isn't enough cheese on their pizza, it goes a long way in the real world.

* My first I.T. job was in an office where the vast majority of the people who worked there had never used a computer at all. Just to stay employed, I had to resist the urge to explain things in a complex way. When I'm trying to sell an idea to a group of people, I do my best NOT to ignore the people in the room who may not understand that idea well. I think that engineers often have a bad habit of getting into engineering arguments with management in the room, where they take things to a level of complexity where management may not understand what's being talked about. Bringing things back down a few levels goes a long way towards getting management to sign off IMHO. Unfortunately, it's a double edged sword, and it can fall flat when management is especially well informed. Classic information asymmetry.

[jnovek]

I would find this behavior extremely aggravating from a co-worker. If you can’t be bothered to edit down your ramblings by hand, just don’t send me anything at all.

[ivell]

Why do we have to insist that messages must be made with lots of effort even if it is hard to understand for the reader? As long as what needs to be communicated is done respectfully, I don't see a value for it to be done manually, especially if the handwritten one is hard to read and thus wasting readers' time.

We don't hold the same standards for spellings. Rather we expect messages to be spell checked before being sent.

[itake]

Maybe you missed my point?

I can either edit down my rambling by hand (costs about 10-30min depending on the length) or I can ask chatGPT for assistance, where I manually edit chatgpt's edits for factual correctness and tone.

--- STT

Like, besides the lease risk, I think 30 to 50% of the business is going to end the moment the owner stops showing up and the motorbikes are gone. Either, I think it was Roger mentioned or you mentioned the Google reviews all mentioned that people go to the bar because it attracts other motorcycle people. And, you know, we don't have an existing, like, network to come in and grow this. And so we might see a 30 to 50% decline in revenue within a month or two with nothing ready to, like, backfill that with. And if our main goal is to make a private club or event space, ideally, I'd like to have some, like, something substantial to, like, commit to that spot. Like, right now, we're three fucking dudes with, you know, a little bit of a vision, but not really. And, you know, we're buying what will be a failing bar unless we, you know, figure out how to run events or use that backspace.

polish

---- gpt5.5

Polished version:

Separate from the lease risk, I think there is a real chance that 30–50% of the business disappears once the current owner stops showing up and the motorbikes are gone.

Either Roger mentioned this, or we discussed it separately, but the Google reviews seem to reinforce the same point: a meaningful part of the bar’s appeal is that it attracts motorcycle people. We do not currently have an existing network or community that can step in and replace that traffic.

That means we could see a 30–50% revenue decline within the first month or two, with no clear plan in place to backfill it. If the main goal is to turn the space into a private club or event venue, I would feel much better if we had something substantial already committed to that location.

Right now, we are three guys with a loose vision, but not much concrete traction. Without a clearer plan for events, memberships, or activating the back space, we may effectively be buying a bar that starts failing the moment the current identity and customer base disappear.

[johnvanommen]

I vote for number 2.

[KronisLV]

> It drives me crazy they can't be respectful of my time and use STT+AI to convert their 90 second monologue to a few written sentences.

I have used Whisper to transcribe audio into text in the past. You could probably build a pipeline for that, whether running locally or in the cloud - and the run the transcription through the same summarization agent.

[adammarples]

Sending me your AI compressed ramblings = straight in the bin

[jamwil]

What did you do prior to 2023?

[tailscaler2026]

Just my two cents: I have coworkers who use AI to drive basically all their communication in Slack and I absolutely hate them with a deep passion. I actively avoid meetings, conversations, and exclude them from everything possible.

If you use AI to drive your communication with other humans, you suck.

[WhitneyLand]

It’s crucial to use for driving/walking.

One problem has been ChatGpt/Claude apps don’t really do this well. They use weak and/or non-reasoning models for voice interaction and the UX is not optimized for hands free.

I wrote an iOS chatbot app mainly for this purpose for myself and family/friends. Allows starting/sending voice prompts with the action button so I never have to look at the screen. Supports any model at any reasoning level so conversations are not dumbed down. Added a video transcription tool so any model can “read” YouTube/Tiktok videos and chat about them. Great to discuss lectures on tech topics.

It takes slightly longer to use a reasoning model for voice interaction use but I prefer the intelligence. The latency can be minimized a few ways, bidirectional streaming helps. It’s TTS agnostic, I’ve got a few selectable providers and the output can be prompt styled “use a chill tone that’s not too eager”.

[gbalduzzi]

What are the use cases of an LLM while walking or driving, that also require high reasoning?

[WhitneyLand]

Most of the problem is that for voice chat, you usually get no reasoning at all and no tool use at all to research or ground assumptions.

For example for voice ChatGPT still uses a quantized gpt40 non-reasoning model that hallucinates pretty frequently. It also doesn’t do much automatic search for updated information and fact checking.

I usually don’t find I need high, usually DeepSeek v4 with medium reasoning is sufficient.

However if it’s important chat like brainstorming on complex topics I sometimes bump it up.

OpenAI has a new voice api that supports adjustable reasoning, but ChatGpt is not using it currently.

[shostack]

With a sufficiently sophisticated harness you can actually do quite a lot by just talking to your AI. I have regularly dictated to build things on my phone while walking to lunch for example.

[WarmWash]

Gemini 3.1 flash live is a native audio to audio model with reasoning. But it's still not a SOTA level model

[jorvi]

I mean, even applied voice 'models' suck for this.

For some godawful reason, Apple Maps voice directions assume that you also understand what it omits. So if it says "turn right in 500 meters" "250 meters" and then you stop at an intersection after 150 meters and it says "turn right", it expects you to understand that it doesn't mean the immediate right at the intersection, but the next one [because you still haven't driven the full 250m]. It is nuts and I have no clue how that has ever gotten past testing.

What it should do is say nothing until I have to turn, or say "turn right in 100 meters" "turn right".

[Melatonic]

This is one thing Waze I think seems to do better than the competition. And they have a ton of different voices.

They also clearly show which voices can do street names (which is hugely helpful). For some reason the Australian and British accented voices feel more polite than the Americans

[mohi13]

How about google maps says "keep north"..as if I am sitting in my car with a magnetic compass...gets my goat everytime

[Ldorigo]

I very begrudgingly started paying for grok for this exact reason. They nailed the voice UI and it works incredibly well with android auto unlike Claude&Gemini (which don't work with android auto at all) and chatgpt (which works well but has hardcored system instructions that make it's voice mode feel like a dopamine deprived Gen Z)

[rob]

I hardly type at all now. I use Handy (free) with Parakeet and use its post-LLM processing feature with a custom prompt tailored towards coding, so I can say things like "Have it go to slash remote dash control" and it'll output "/remote-control". Converts brackets, etc.

Everything is almost instant, it's insanely fast, and lets me work on multiple different agents/windows at the same time fast with cmux.

I use the same thing to talk to people on Slack, iMessage, etc now when I'm working from home instead of typing.

I also can help articulate my thoughts better when I'm thinking them literally out loud instead of just sitting silent and typing them on a computer for hours.

It's just something that you need to try and get used to because I also thought it was something I wouldn't like at first.

[thefreeman]

Can you share more information on the post-LLM processing and the prompt you use? I would like to try this out but don't see any post-LLM options in Handy.

edit: nevermind, found info on the docs about how to enable post processing. Would still be interested in your prompt though if you don't mind sharing!

[rob]

You have to enable "Experimental Features" under "Advanced."

This is the prompt I use (it's probably overkill and can be condensed):

https://pastebin.com/raw/RUVAqLCU

[ezconnect]

What is Parakeet?

[rafaelm]

I believe this is the correct link. I use it too in Handy, for English and Spanish transcriptions: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3

[stijnveken]

Maybe they meant narakeet?

https://www.narakeet.com/tools/

[dghlsakjg]

Parakeet is the name of a speech to text model from Nvidia. Roughly comparable to whisper from openAI.

It's the model doing the work inside the wrapper that an app provides.

[rob]

Yep, here's the v2 and v3:

https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2

https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3

It's almost instant on my new M5 Max w/ 36GB of memory, but I used both with Handy on my previous 2019 Intel Mac w/ 16GB memory and was completely surprised at just how fast it was for being on-device! Not instant, but only a couple seconds.

[weitendorf]

I thought this way until I tried it, and the main difference is that when I'm managing tons of agents at once or just reviewing some plan / approving next steps, or need to give quick feedback/ask a simple followup, the voice interface makes me much faster and more likely to continue because it's lower friction (and in many cases that's good, though not all) and can be hands-free.

Actually, my thoughts on this matter changed so much that it inspired me to get much more into voice controls because I realized how this same problem was basically why some people sucked at remote work or weren't able to properly use tools like claude code, because it was essentially the same problem but worse (typing / messaging feeling too high-friction or raising the barrier for participation). I have a way to let Claude call me now to tell me stuff when I have a bunch of instances out doing stuff and then leave to go home.

I'm trying to get that better integrated in my devloop because I think it makes managing >4 agents simultaneously much more feasible and natural for some people (I used to play Starcraft a lot so I'm used to the multitasking, but it still takes sustained willpower to be constantly "driving" or monitoring things, or to field questions), especially ones who have never served as TLs or people managers before. IMO it's a big performance roadblock for a lot of developers to be treat directing multiple agents simultaneously as some kind of high-stakes/high-cost thing. The kind of developer who would not say anything in a team meeting unless prompted or who thinks everything is stupid by default (because they are afraid of making decisions / being wrong even if only briefly) is both very common and reluctant to work this way, but also really probably needs it to be as productive as more skilled developers.

[noduerme]

I don't know about you, but I force myself to read the whole spaghetti thought process of any AI that's actually working on code, and make sure I understand what the hell it just said before I ask questions or give it a green light. Even or especially when whatever it said is full of fluffy stuff about having understood the problem space. That's usually where a well-placed question can bring the entire structure crashing down.

"You're right to push back" has become the gold standard phrase I'm looking for from these things to assure myself that I'm covering all the bases and understanding what it's building (not that that's enough, and not that it isn't still going to build some ungodly blob anyway).

I kinda like using voice to jot down my next questions or iterate on things, but there's a clear danger to it, which is that you may inadvertently be signing off on stuff you haven't thoroughly read. If there's one thing about LLM-written code, it's that the devil is in the details.

[NikolaNovak]

I type as fast as I talk so for majority of my LLM usage I don't need text to speech.

But I love the chatgpt voice interface e.g. on a long drive when I can use it to learn about random stuff (btw, turn advanced voice off for such usage).

Other part though is, hacker news vs regular population, majority of which would much much rather talk and listen than type and read.

[kingkongjaffa]

I like to talk (stt) but I don't want tts to talk back to me I just want to read the response. voice synthesis is a waste for me personally.

[perching_aix]

It's easier, faster, and more natural to talk than to type for the vast, vast majority of people.

This trivial fact of life is observed every day by e.g.:

- students taking notes and finding it necessary to only jot down key facts so that they can keep up,

- stenographers who require special training and equipment to keep up verbatim with live speech in the courtroom,

- annoying colleagues who insist on "hopping on a quick call" or arranging big, wasteful, and disruptive meetings instead of just writing down their problem / sending a message or email,

- friends who insist on sending short voice messages in DMs instead of typing, because it's more "personal" that way (which to be fair it is, but not to the extent proclaimed).

[QuantumNomad_]

When I was still using OpenAI, I used it among other things to translate from English to Spanish while talking to Spanish-speaking people in person.

I understand a bit Spanish but I don’t speak Spanish yet, and they don’t speak English.

I speak English to the AI and end with “translate to Spanish, translation only”, and then the AI says the thing I was saying in Spanish (not perfect but good enough, and also it has a slightly weird accent that might be it using English or English influenced text to speech even when speaking Spanish sentences?).

[pid-1]

I've been using ChatGTP by voice for things like cooking and house repair stuff. It's quite convenient for situations in which your hands are busy.

Other week I fixed a a water valve. After planning the thing with ChatGTP I brought the new valve. Then I described what I was seeing as I swapped the old valve for the new one to make sure everything was right. Really cool experience!

[justech]

Faster, and that's it. If you don't need precision (like with prompting LLMs) the speed gain is massive (*for most people)

[fdsjgfklsfd]

Sometimes it's faster than swyping on a phone, but mostly I use it to learn about stuff and hash out ideas while driving.

[noduerme]

This may sound strange and even callous, but I think it's appealing to people who are used to having employees. It's not about speech being a better interface, it's that thinking hard enough to sit down and compose a prompt is too much work if you're used to just yelling at someone.

Pity the managers with no one left to boss around besides the machines coming for their own jobs.

I was asked just yesterday if I could wire up [redacted] so that [redacted profession] could have a realtime voice interface while in the middle of performing [redacted]. My basic answer was yes, but it would be a bit slower than you want if something is going wrong, and it would probably be unethical for a whole lot of reasons.

[stranded22]

Accessibility.

[noduerme]

What about accuracy?

[limflick]

I'd imagine it'd be a reasonable tradeoff for disabled people who can't use their hands.

[arcanemachiner]

Much faster and better flow. Don't knock it til you've tried it.

[throawayonthe]

it's very confusing. maaaybe if the stt is good and fast enough, speaking may be faster? english speakers can probably hit 150-180 wpm but seems like a hassle

[hidelooktropic]

I can talk faster than I can type.

[emodendroket]

A lot of people are slow typists.

[greenavocado]

Also vision can be used for "compaction" https://blog.can.ac/2026/06/10/snapcompact/