[dahwolf]

A lot of people are projecting for ChatGPT to come for the search market, but I wonder how that will play out for lowly cognitive queries.

Quite a few people use search by awkwardly typing on mobile one or two words, probably misspelled and/or auto-completed as they type it. The query isn't sophisticated, lacks a lot of context and parameters, which the search engine then tries to guess.

When you use ChatGPT in that way, you'll get useless generic answers. It seems to shine specifically when being more specific, detailed, which also suggests users are willing and able (education level) to give such rich input.

The idea that it's better than search for this specific normy behavior, I openly question. And let's not forget about the economics. More expensive to run, vastly less ad space, and content owners (the whole web) are going to be pissed and will put up ever higher walls.

[bigyikes]

Just wait until we have a model that automatically translates vague, awkward prompts into something more useful.

Put differently: if Google’s search models already have the ability to return great results for poor queries, why couldn’t a large language model (or a plug-in for one) learn the same algorithm?

[unshavedyak]

> When you use ChatGPT in that way, you'll get useless generic answers. It seems to shine specifically when being more specific, detailed, which also suggests users are willing and able (education level) to give such rich input.

Sidenote, i've found GPT useful enough to pay for (GPT Plus) by doing the opposite. Or rather, i find it very useful when i struggle to search for problems. ChatGPT helps guide me to new search or research terms, sometimes even providing the answer more directly.

It feels like the olden days where Google was great at finding a movie based on some vague movie description. GPT does that for a ton of things for me, enough that i found it useful.

It hasn't replaced online research but it has accelerated it for me.

[LASR]

What people forget is the underlying capability - LLMs are able to do reasoning.

So the one-track thinking of garbage-in-garbage-out is not the limitation any more.

What we're precisely now able to do is garbage-in-less-garbage out.

You can take a vague prompt in and ask GPT to hypothesize on what it means, why the user is asking that question and then generate a detailed prompt. Then use that that prompt and then perform the search.

Trying this out in the playground, I see a suprisingly capable search experience.

[barking_biscuit]

This kind of second-order (and higher-order) usage of LLMs is where things actually start to get much more interesting. The other thing you can do is just train a better model.

I use GPT-4 for debugging a lot now, because it's excellent at taking nothing other than an error message from the console and giving me back what's wrong and how to fix it. It's not perfect, but it's good enough that I reach for it by default now. I don't have API access to GPT-4 yet, and so I was comparing how well GPT-3.5 performed at this same task and for the example I tried, it just didn't get close enough for me to truly find it useful, so I wouldn't begin to rely on it in my daily workflow unlike GPT-4.

But... what I am actually quite interested in, and what I'm seeing a lot of, is exactly how far can you push a less capable model through prompt engineering? I think it's actually surprisingly further than you might have initially thought.

[okasaki]

Have you tried it?

I just typed "eli5 hn vs redit" (misspelled reddit), and it understands perfectly.