Good to know it can do that, in the pasted chat above it didn't. To be honest, it surprised me it couldn't, this isn't exactly a very hard guess given the computation results. It doesn't convince me GPT4 is anywhere close to winning the IMO, though :-)
True. I let it continue and even prompted it with hints but it cannot find the proof that non-prime powers fail. It can however prove prime powers work.
It's a good point. The knowledge cut-off says Jan 2022, but in the past there's been reasons to suspect some data after the cut-off is in the model too. There's also the possibility that it has seen the problem somewhere else before Jan 2022.
Nevertheless, I think the more important question is whether GPT-4 is capable of
1) Listing all solutions less than 100
2) Figuring out the commonalities of the solutions
For 1) I have no doubt that the answer is yes based on its coding skills, in fact it is much stronger at coding than this. For 2) my subjective feeling in playing around with it is it's not consistent at similar problems but it can do it sometimes. Maybe in this case it has seen the list of prime powers <100 so it's very easy for it.