But in this case it's not like that at all. They only saw the NAME of the problem. Like if I said "Page 23 of Mathbook Y, problem number 3". Which happens to be 6x6.
I know this is deep down a bad comment thread, but I thought I'd chime in here.
I have been writing function names and test names, and then telling gpt to fill in the test, which is usually does how I want (maybe with errors, but it tests the correct thing), and then I tell it to fill out the answers.
this is in a thing I'm building that's never been built, with names that I made up (but describe the functionality well)
It cannot have this spot memorized, I just invented it myself
If I gave you a programming problem and all I told you was that the problem name was Traveling Salesman, you might be able to solve it based on that.
If not that, then if I just said "fizzbuzz" to you, I'm sure you would be able to give the solution without me needing to say any other descriptions of the problem
But in that case, not memorization of the specific problem set, but "programming background knowledge." Hardly something to blame the machine for when we rely on it every day.
It said they gave the exercise name, which doesn't sound like just the exercise number but probably mildly descriptive -- and they also gave it function stubs.
I have been writing function names and test names, and then telling gpt to fill in the test, which is usually does how I want (maybe with errors, but it tests the correct thing), and then I tell it to fill out the answers.
this is in a thing I'm building that's never been built, with names that I made up (but describe the functionality well)
It cannot have this spot memorized, I just invented it myself