[the_other]

You write as if you’ve found a hole in the article’s argument. The lack of evidence is a hole in the reporting, for sure. The tone of your comment suggests you feel that by not publishing all their evidence, the author’s point is wrong (rather than under-justified). However, the example you use to back up your point also backs up the article’s point. The article’s point is that ChatGPT doesn’t summarise, it only shortens. Your example indicates shortening, but not summarising.

[jonplackett]

There’s just so many articles of people whining about how ChatGPT can’t do things, when they clearly havent prompted it very thoughtfully.

So I think that’s why you see so many reactions like this.

I’ve found chatGPT incredibly good at all sorts of things people say it is bad at, but you need patience and to really figure out the boundaries of the task and keep adding guidance to the prompt to keep it on track.

[merlincorey]

The article makes it clear that there is a semantic difference between shortening and summarizing and that importantly summarizing requires understanding which ChatGPT most certainly does not have.

One example in the article is that if you have 35 sentences leading up to a 36th sentence conclusion, ChatGPT is very likely to shorten it to things in the earlier sentences and never actually summarize the important point.

[arisAlexis]

which chatgpt ?

[totallymike]

It doesn't matter which. The concept of understanding is entirely orthogonal to what an LLM is and how it works. It has no such thing, and can't.

[arisAlexis]

You seem to be on the "statistical next token predictor" side. I'm more.on the side of those who invented it (they should know) that think these machines can understand things

[goopthink]

In 1964, Joe Weizenbaum created a chatbot called "Eliza" based on pattern matching and repeating back to users what they said. "He was surprised and shocked that some people, including Weizenbaum's secretary, attributed human-like feelings to the computer program." People are notorious for anthropomorphizing and attributing to things attributes (including human-like attributes) that they do not possess. [1,2] LLMs are a "statistical next token predictor" by their design. The discovery that coherent and interesting communications are relatively easily statistically modeled and reconstructed if given enough computing power and corpus of training data does not therefore imply that these programs have latent thinking and understanding capabilities.

Just the opposite: it calls into question if _we_ have thinking and understanding capabilities or if we are complicated stochastic parrots. [3] The best probing of these questions is done at the limits of comprehension and with unique and previously unseen information. I.e., how do you comprehend and process to previously unseen/unfelt/not-understood qualia? Not about how you deal with the mundanity of reactions between people (which are somewhat trivial to describe and model). [4]

[1] https://en.wikipedia.org/wiki/ELIZA [2] https://en.wikipedia.org/wiki/Anthropomorphism [3] https://www.newyorker.com/humor/sketchbook/is-my-toddler-a-s... [4] https://en.wikipedia.org/wiki/Games_People_Play_(book)

[jonplackett]

In other news, someone hits a piano five times with a hammer and proclaims pianos are no good at making music.

[jules-jules]

At what point does it become easier to just do the task yourself? I’ve pondered this question often and came to the conclusion that it’s not worth at the current level of output for me to tinker with it until I get sensible responses.

[jonplackett]

It depends on the task. Sometimes I have just given up when it really can’t get something.

But other times I’ve persevered and once it’s ‘got’ it, it can then repeat it as many times as I need. That’s the knack really. Get it to the point of understanding and then reuse that infinitely and save yourself a lot of time.

[kibbi]

In the example I mentioned, ChatGPT 4 did keep all essential statements of my texts when reproducing shorter versions of them. For example, it often wrote one high-level sentence which skillfully summarized a paragraph of the original text. As far as I understand, this is what the author meant by 'summarizing' vs. 'shortening (while missing essential statements)'.

I was impressed at those high-level summaries. If I had assigned this task to several humans, I'm not sure how many would have been able to achieve similar results.