I was prompting it along this line of thought earlier. What I found was that it doesn't seem like it can do anything novel, which is to be expected, but I can see myself working with it to discover novel things.
Sure, I agree there - but the point is it cannot understand code. It can try to describe it, but it isn't able to reason about the code. You won't be able to coax it to the correct answer.