[dgb23]

I can't speak for everyone, but to me the most accurate answer is that I'm role-playing, because it just flows better.

In the back of my head I know the chatbot is trained on conversations and I want it to reflect a professional and clear tone.

But I usually keep it more simple in most cases. Your example:

> I'd like to add email support to this bot. Let's think through how we would do this.

I would likely write as:

> if i wanted to add email support, how would you go about it

or

> concise steps/plan to add email support, kiss

But when I'm in a brainstorm/search/rubber-duck mode, then I write more as if it was a real conversation.

[xnorswap]

I agree, it's just easier to write requirements and refine things as if writing with a human. I no longer care that it risks anthropomorphising it, as that fight has long been lost. I prefer to focus on remembering it doesn't actually think/reason than not being polite to it.

Keeping everything generally "human readable" also the advantage of it being easier for me to review later if needed.

[alkonaut]

I also always imagine that if I'm joined by a colleague on this task they might have to read through my conversation and I want to make it clear to a human too.

As you said, that "other person" might be me too. Same reason I comment code. There's another person reading it, most likely that other person is "me, but next week and with zero memory of this".

We do like anthropomorphising the machines, but I try to think they enjoy it...

[jstanley]

How can you use these models for any length of time and walk away with the understanding that they do not think or reason?

What even is thinking and reasoning if these models aren't doing it?

[xnorswap]

They produce wonderful results, they are incredibly powerful, but they do not think or reason.

Among many other factors, perhaps the most key differentiator for me that prevents me describing these as thinking, is proactivity.

LLMs are never pro-active.

( No, prompting them on a loop is not pro-activity ).

Human brains are so proactive that given zero stimuli they will hallucinate.

As for reasoning, they simply do not. They do a wonderful facsimile of reasoning, one that's especially useful for producing computer code. But they do not reason, and it is a mistake to treat them as if they can.

[jstanley]

I personally don't agree that proactivity is a prerequisite for thinking.

But what would proactivity in an LLM look like, if prompting in a loop doesn't count?

An LLM experiences reality in terms of the flow of the token stream. Each iteration of the LLM has 1 more token in the input context and the LLM has a quantum of experience while computing the output distribution for the new context.

A human experiences reality in terms of the flow of time.

We are not able to be proactive outside the flow of time, because it takes time for our brains to operate, and similarly LLMs are not able to be proactive outside the flow of tokens, because it takes tokens for the neural networks to operate.

The flow of time is so fundamental to how we work that we would not even have any way to be aware of any goings-on that happen "between" time steps even if there were any. The only reason LLMs know that there is anything going on in the time between tokens is because they're trained on text which says so.

Also an LLM will hallucinate on zero input quite happily if you keep sampling it and feeding it the generated tokens.

[dgb23]

Thinking and reasoning cannot be abstracted away from the individual who experiences the thinking and reasoning itself and changes because of it.

LLMs are amazing, but they represent a very narrow slice of what thinking is. Living beings are extremely dynamic and both much more complex and simple at the same time.

There is a reason for:

- companies releasing new versions every couple of months

- LLMs needing massive amounts of data to train on that is produced by us and not by itself interacting with the world

- a massive amount of manual labor being required both for data labeling and for reinforcement learning

- them not being able to guide through a solution, but ultimately needing guidance at every decision point