[jvanderbot]

Inb4 we all tell you you're using it wrong. There are certainly better ways to do this, but this is really the major thing that drives me nuts. No matter how many times I tell some LLM I want it to ask clarifying questions to provide better answers, it just won't. You end up doing exactly this, guessing what information might trigger the right recall.

[arcanemachiner]

I've found that there are often different trigger phrases I can use to get an LLM to "change its mind".

For Llama, I can just say "Are you sure?" and it will change its tune (unless it's quite "certain" about the results).

Qwen is more insistent, but will change course if I say "I looked it up and it says you're wrong".

[Avicebron]

if we formalize what might trigger the recall we could keep a mental model of it, maybe call it bangs, something like !wikipedia would only index data sourced from training on wikipedia..you might be on to something..

[BoorishBears]

> No matter how many times I tell some LLM I want it to ask clarifying questions to provide better answers, it just won't

You're telling me there's a modern frontier model that refused to ask clarifying questions after you told it to?

[jvanderbot]

I can't tell if you're being sarcastic, but yes, when it asks questions they are trivial disambiguation of mainly language, and never information seeking like you might expect when doing more than superficial investigation

[BoorishBears]

If you can't tell if I'm being sarcastic about an LLM asking clarifying questions, you are possibly not great at using LLMs.

Prompting isn't necessarily the career some people wanted was sold as, but it's not a bad idea to practice a bit and build a sense of what a clear and effective prompt looks like.

-

To be clear, I get telling people it's a "you" problem every time an issue with LLMs comes up isn't helpful... but sometimes the disconnect between someone's claimed experience, and what little most people actually can agree LLMs are capable of is so great that it must be PEBCAK.

I just tried the original checkpoint of GPT 3.5 Turbo and it was able to handle drilling up and down in specificity as appropriate with the prompt: "I need you to help me remember a movie. Ask clarifying questions as we go along."

[BehindBlueEyes]

I think it depends a lot on what llm and tooling you have access to.

I had great results prototyping agents at work for specific tasks and answering in a specific style including asking appropriate clarification questions.

But at home with just free/local options? I've nowhere near the same settings to play with and only had very mixed results. I couldn't get most models to follow simple instructions at all.

[jvanderbot]

Nobody has more PEBKAC problems than me. It might be my experience is colored by older models. I can give it another shot. But I do use LLMs quite a lot and got decent and surprising functionality out of them in the past. I was just consistently vexed by this one thing when doing information seeking search -like activity.