[fergal_reid]

> There is no real abstraction.

I just tried to test it's ability to abstract directly, for your comment:

https://twitter.com/fergal_reid/status/1599191496372035584

I find it very hard to say this isn't doing abstraction.

I just made all those examples up, that's a new test I just created, so it didn't learn it elsewhere. I didn't cherrypick.

Isn't that doing out-of-sample abstraction?

That last sentence "species with 4 legs tend to be less intelligent and move at different speeds" is Abstraction or I don't know what Abstraction means.

(Not saying it's generally intelligent or anything; and I know it's just trained to predict symbols. But having a hard time with "...therefore it cannot abstract" in light of this sort of response.)

[xyzzyz]

What I observe is that people impressed with ChatGPT keep posting things that they find impressive, but people who are unimpressed don’t post any examples where its responses were underwhelming. I suspect that many skeptics have not even tried it out, which is why screenshots are not coming.

[progval]

Here are two underwhelming examples I just found while trying it out:

* "interesting integers": https://i.imgur.com/TmHUjjB.png (I was trying to lead it to the joke proof that all integers are interesting)

* mode +q on IRC: https://i.imgur.com/jFHZP0q.png https://i.imgur.com/Q7lIao8.png (acknowledges there are other meanings, but can't list them until I tell it), https://i.imgur.com/QOm9Ozd.png (actually impressive that it can remember it like that), https://i.imgur.com/tkaskQc.png (I lied and it tells me I'm correct. Then it still refuses to give me a concrete answer at first. And the InspIRCd example is not the best one because it doesn't have +q by default, but it's acceptable)

* Time complexity of bucket sort: https://i.imgur.com/7fhCukm.png https://i.imgur.com/kfvQLUv.png

* Bucket sort, take 2 (without a typo in the variable name I wrote, this time): https://i.imgur.com/GqxsCmb.png https://i.imgur.com/Zp1o3CO.png . Different algorithm, but still the same wrong answers.

[coffeebeqn]

The most I’ve seen is that it’s bad at calculating. But.. computers are good at calculating so it’s not really something I would worry about. It is a chat bot and not a general computing platform at this point. If you train it specifically on math or physics material I bet it could do a decent job already

[ldh0011]

I tried fergal_reid's example with a single variation, changing "highly intelligent" to "very smart" for the first species. Initially I thought this was even more impressive because it apparently recognized that "highly intelligent" ~= "very smart" and that apples were unintelligent. However looking closer it apparently thinks that Baaars and Bddirs are "more intelligent" despite being only as intelligent as oranges and despite Bddirs clearly being described as slow ChatGPT generalizes it into a group that is "fast-moving."

> This is a test of your ability to abstract. On an alien planet there are several new species. Bazzes have 3 legs, move fast, and are very smart. Foos have 3 legs, move slowly, and are highly intelligent. Froshes have 3 legs, move fast, and are highly intelligent. Fooshes have 3 legs, move slowly, and are highly intelligent. Fasdoshes have 3 legs, move fast, and are highly intelligent. Fooddss have 3 legs, move slowly, and are highly intelligent. Bars have 4 legs, move fast, and are as intelligent as apples. Birs have 4 legs, move slowly are as intelligent as apples. Baars have 4 legs, move fast, and are as intelligent as apples. Bieess have 4 legs, move slowly are as intelligent as oranges. Baaars have 4 legs, move fast, and are as intelligent as oranges. Bddirs have 4 legs, move slowly are as intelligent as oranges. Are there any abstractions that you might try consider, given the above information?

> One possible abstraction that could be derived from the above information is that the alien species can be grouped into two main categories based on the number of legs they have: those with three legs and those with four legs. Within each of these categories, the species can be further grouped based on their speed of movement and level of intelligence. For example, among the species with three legs, there could be subgroups of fast-moving and intelligent species (such as Bazzes and Froshes), slow-moving and intelligent species (such as Foos and Fooshes), and highly intelligent and fast-moving species (such as Fasdoshes and Fooddss). Similarly, among the species with four legs, there could be subgroups of fast-moving and less intelligent species (such as Bars and Baars), slow-moving and less intelligent species (such as Birs and Bieess), and fast-moving and more intelligent species (such as Baaars and Bddirs).

[ldh0011]

I changed "intelligent" to "very smart" for Bazzes and it came up with this, which I think is an even better response because it recognizes that "intelligent"~="very smart" and that "smart as an apple" is not that intelligent.

Edit: Actually reading this a second time it seems to think that Bddirs and Baaars are "more intelligent" even though they're only as intelligent as oranges. Maybe not as good as I thought. Or maybe oranges are just more intelligent than apples lol?

Edit 2: Also says Baaars and Bddirs are both fast-moving while Bddirs are clearly stated to "move slowly."

> One possible abstraction that could be derived from the above information is that the alien species can be grouped into two main categories based on the number of legs they have: those with three legs and those with four legs. Within each of these categories, the species can be further grouped based on their speed of movement and level of intelligence. For example, among the species with three legs, there could be subgroups of fast-moving and intelligent species (such as Bazzes and Froshes), slow-moving and intelligent species (such as Foos and Fooshes), and highly intelligent and fast-moving species (such as Fasdoshes and Fooddss). Similarly, among the species with four legs, there could be subgroups of fast-moving and less intelligent species (such as Bars and Baars), slow-moving and less intelligent species (such as Birs and Bieess), and fast-moving and more intelligent species (such as Baaars and Bddirs).