Hacker News new | ask | show | jobs
by WhitneyLand 20 days ago
Much better than it was before but it’s still significantly weaker than a direct chat.

For example I asked

“Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?

The voice response was directionally right but lacked detail and was a little hand wavy.

The answer to the same question in a text chat was much higher quality.

The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.

I had reasoning set the same for both.

3 comments

A lot of this is steerable! Our current personality is optimized for brainstorming and conversations, but you can provide custom instructions to ask it to go deep and give you info-dense or more technical answers.
I found myself probing more and more questions to have it do that, I wish it could remember to provide details answers, I thought the replies were way too short and hand-wavy like one of the prior comments suggested.
That would be cool, but is there a way to differentiate between voice mode and text only output in the custom instructions?

Otherwise, it seems like preferences could be competing.

You can read far more text than listening. If the voice response is too long, people lose the patience quickly. So it is better to show it on screen if we have too much text.
I think the bare truth is that the target audience for this product is not people who are highly particular about terminology in answers involving vector mathematics.

It’s a different set of tradeoffs for users that don’t already have strong engagement or interest in existing AI products.

Do you realize how many more people prefer to chat over the phone and watch television or videos in their free time vs type multiple paragraphs of text into a chat window and then read 3x more back?

I’m not even talking about grandma here, it’s a non starter for the vast majority of humans who don’t spend their free time writing and reading tech news. To most people, having to write out a bunch of words describing their problem/goals, then sift through pages and pages of detailed response to get an answer, feels overwhelming and not worth doing.

I would naturally assume they cut the background thinking level to the minimum. It's a halting problem question and with in text 5.5 with max thinking will chew on a question for 5-10 minutes sometimes. That would make a pretty awkward conversation.