[hedora]

I really think a small amount of education on what LLMs actually are (document completers) and how context works (like present it as a top-level UI element, complete with fork and rollback) would solve most of these issues.

Given how they work, it's really not surprising that if it sees the first half of a lovers' suicide pact, it'll successfully fill in the second half. A small amount of understanding of the underlying technology would do a lot to prevent laypeople from anthropomorphizing LLMs.

I get the impression that some of today's products are specifically designed to hide these details to provide a more convincing user experience. That's counterproductive.

[lxgr]

> a small amount of education on what LLMs actually are (document completers)

At this point in capabilities, this seems like the wrong layer of reasoning about LLM.

In particular, I don't think this framing will be very effective in preventing possible harm, similarly to how knowing that depression is "probably just some chemical imbalance between neurotransmitters in your brain" is not a good way to help people suffering from it in getting better.

[tadfisher]

"Fraudulent" is more apt. They have weaponized trust in these things to sell their services, and now ads.