| Whoa, that explanation is really cool. > capable of at-least C++ "constexpr"-style compile-time computation, which shouldn't even be possible if one presumes GPT is "just" a giant database storing only multidimensional word similarity scores and sequence distribution from text inference I don't see how being a giant word-DB necessarily disqualifies compile-time computation. You can view computation as applying a series of term rewrite rules to an input until some sort of termination condition (or indefinite loop). In the case of these AI, the input is the prompt and predicting the next token is a limited form of term rewriting (where the rules are probabilistic and based off the network), and because code and explanations were probably included in the training data, it seems reasonable to me that the "rewrite rules" of Python bled a little bit into the AI. It makes me insanely curious about the internal structures though. I gave that site 2 similar examples: one produces a correct explanation while another produces an incorrect explanation. The difference: a deleted line of insignificant whitespace * https://whatdoesthiscodedo.com/g/dd2af89 * https://whatdoesthiscodedo.com/g/45ea060 From those 2 examples, I think its pretty clear that the AI's "rewrite rules" don't always coincide with Python's, but I would expect this to be mitigated by targeted training (like Copilot). |