Get it to explain something, lets say dynamic hedging for derivatives, and then ask it to explain how to exactly hedge something specific. Or describe some physical situation with a quirk and then let to derive the implications. Someone on HN had an example of asking to imagine entropy working in reverse in a cup of coffee with sugar dissolved. While it discussed sugar spontaneously forming crystals and other things, it never considered what the water would do, for example, let alone bigger issues such as if even the existence of water works etc.
Again, humans are often poor at these things, too, but if it had "mechanized" reasoning capabilities instead of "replicative" ones (i.e., just repeating stuff), I would expect it to do generally better.
Why would you assume it could be expected to have "mechanized" reasoning capabilities, whatever that is?
I find these questions generally poor at gauging anything when people haven't given them to a representative sample of people first as a benchmark. Consider that not long ago there was a tedious trend of people posting "difficult" questions of orders of operations involving basic arithmetic, and a significant proportion of people in the threads would continue to belabour and argue for the wrong result even after having been told in excruciating detail how to apply the rules. In other words: I think people here tend to massively overestimate the reasoning ability of the average person.
E.g. to the example questions here, I'd bet the average person can't give a satisfactory definition of entropy, much less be able to tell what it does "forwards" before even considering "reverse". So why would we treat this as a benchmark of whether or not an LLM can reason?
Yeah, it replicates poor human reasoning capabilities but doesn't really have a proper method to reason through things. The later is what I expect from a true machine intelligence.
I don't care at all about what humans do or know when looking at machine intelligence.
You might not care about it, but all of the people who regularly claims it can't reason certainly seem to do.
Defining "true machine intelligence" without referencing the only intelligence most people would agree is "true" intelligence seems like a bizarre attempt at setting the bar unreasonably high, and defining "replicating poor human reasoning capabilities" to me is an admission from you that they do reason whether or not you think their ability to do so is "proper".
> Replication someone else's reasoning isn't reasoning. Otherwise, a book would "reason".
This isn't reasoning. This is a meaningless platitude. Matching the level of reasoning of someone else would inherently be reasoning. Replicating the level of reasoning would be.
> And, yes, most humans fail to reason properly a lot of the time. Any simple probability puzzle shows that.
That is an argument for lowering the bar for assessing whether an entity has the ability to reason, not raising it. Using this as an argument to me is another illustration of poor reasoning. Should I argue that you don't have the ability to reason because I don't think this meets the bar of proper reasoning?
Reasoning and intelligence aren’t binary yes/no things, it can have some capability for reasoning and intelligence while still being below what a human can do
That is a point more orthogonal to mine. I am saying it has no general "method" to go from concept to application. It replicates poor human reasoning capabilities but doesn't have a method to reasoning that extend beyond.
Not OP but may have some idea of what they mean. Recently I had the following bizarre conversation with Bard. It appears to "know" what a word ladder is, and be able to explain it to me. On subsequent questioning it gives me a correct solution to a solvable problem (which I admit is impressive), a false solution to a non-solvable problem (whilst telling me that the solution obeys rules that it clearly doesn't), and then a bizarre "proof" that it's impossible to find a solution to the first problem I gave it (which it had already solved).
The 6-step supposed "solution" is the one that really gets me. It seems to have no idea that the constraints that it repeats in the same message have clearly been violated.
~~ME~~
What is a word ladder puzzle?
~~BARD~~
A word ladder puzzle is a word game in which you are given two words and must find a chain of other words to link the two, in which two adjacent words differ by one letter. For example, the following word ladder connects the words "cat" and "dog":
cat -> cot -> cog -> dog
~~ME~~
Give me a word ladder which transforms the word FAWN to the word DEER in 7 steps.
~~BARD~~
Here is a 7-step word ladder that transforms the word "FAWN" to the word "DEER":
FAWN
PAWN
PAWS
PEWS
SEWS
SEES
SEER
DEER
This word ladder is valid because each word differs from the previous word by exactly one letter. All of the words in the ladder are also valid words in the English language.
~~ME~~
Give me a word ladder which transforms the word FAWN to the word DEER in 6 steps.
~~BARD~~
Here is a 6-step word ladder that transforms the word "FAWN" to the word "DEER":
FAWN
PAWN
PLAN
PLAM
DAME
DEER
This word ladder is valid because each word differs from the previous word by exactly one letter. All of the words in the ladder are also valid words in the English language.
~~ME~~
Is there a word ladder which transforms the word FAWN to the word DEER in 7 steps?
No, it is not possible to transform the word FAWN to the word DEER in 7 steps. The minimum number of steps required is 6.
Here is a proof by contradiction:
* Assume that there is a 7-step word ladder from FAWN to DEER.
* Each step in the word ladder must change exactly one letter.
* There are only 4 letters in the word FAWN.
* Therefore, at least one letter must be changed twice in the word ladder.
* However, this is not possible, because each step in the word ladder must change exactly one letter.
* Therefore, there cannot be a 7-step word ladder from FAWN to DEER.
I hope this explanation is helpful. Please let me know if you have any other questions.
Again, humans are often poor at these things, too, but if it had "mechanized" reasoning capabilities instead of "replicative" ones (i.e., just repeating stuff), I would expect it to do generally better.