[Hasz]

Counterpoint, I think this is true for some archetypes of people, but certainly not everyone. I personally use it like the socratic method. I am an intermediate user, I spend a ton of time with LLMs at work and personally, both prompting and letting some crappy agents try to automate boring work. I primarily use Gemini and ChatGPT models, along with some Chinese smaller weight models (eg qwen) locally.

If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.

[abustamam]

This, I will use Obra Superpowers brainstorming skill to propose/refine a few viable solutions for a feature or bug I'm trying to solve. After it asks me clarifying questions and presents a spec, I will say "well what about X or Y". The I'll run the grill me skill on the spec to tighten it up, clarifying any assumptions made.

I find it to be a really tight loop and results in very high quality code at a high velocity.

[nsagent]

My two modes of using LLMs has been to try it for 1) natural language search queries where traditional search engines have failed and 2) occasionally as a sounding board using the socratic method.

Inevitably, it fails frequently at both. Any "reasoning" it is doing is merely rehashing ideas that someone else has already posited. This helps some of the times, but the vast majority of the time it just chooses a biased perspective (frequently the most common) and then regurgitates tired old talking points. This contrasts greatly to speaking with others who often have more intuitive notions that tend to be less polished and rote.

I'd love for LLMs to be better sounding boards, but so far they fail miserably far too often for my tastes. To each their own though.

[jubilanti]

Are you also one of those people who believes that advertising works on other people, but not you?

[Hasz]

Advertising is fine (not great), especially when highly targeted and relevant. Spam, misleading, or predatory advertising is not.

[greenavocado]

> If you treat the model like an excellent bluffer, it has never been more fun to challenge a model. To me, there is something deeply intellectually satisfying about "proving" it incorrect, and I like being deeply critical of what the model spits back out. I find that refinement process (with the constant sycophancy turned down in the system prompt) creates a really good loop of critical evaluation that would be hard to get in anywhere else. You can treat it just like the Socratic method, but instead of a benevolent teacher, you get a probabilistic bullshit artist. Lots of fun, highly recommend.

Yes, but eventually the intellectual whack-a-mole gets tiresome unless you get really, really good at simultaneously cornering it and not letting it concede to your point.

[Hasz]

new session. It's easy to lead a model into getting the response you want, deliberately or accidently.

The point is not to literally win an argument (it doesn't matter), it is to use the model like a partner to poke holes in your own understanding. Once it's poked a hole, it has served its purpose. Plus, you eventually run out of context or the model trails off into babbble.

[visarga]

That is how I use models too. Sometimes I have them role play the supporters of the idea I am arguing against.