| It is interesting to me how controversial this post is. It has the highest upvotes, and most disagreeing comments, of anything I've typed up on HN. I'll gladly admit I think what these companies are doing is unethical, and I'm sure that biases my thinking toward skepticism. That said, there remains way too much that is hidden to be able to effectively evaluate what is going on. You have the perfect storm: - AI companies do not share their custom internal harnesses.
- AI companies do not share their custom internal training data.
- AI companies do not share how much compute they allocate to trying to solve problems of this nature.
- AI companies are primarily marketing their models to investors as human-replacing rather than human-augmenting.
- AI companies are under enormous financial pressure to make their business work.
The last two points incentivize them to find these types of "first proof" successes as aggressively as they can, and I'm sure they've thrown the whole book at it.Is it likely that they literally had a mathematician discover this, put it into the training data, and then prompted it out? Of course not. But it would make a world of difference--in evaluating the impressiveness of this discovery and LLM capabilities in general--if we were to know the extent to which the training data crosses over this problem, the harness with which this was ran, and how much compute was spent. Until they bring more transparency to the whole process--something which some of the mathematicians commenting on this even asked for--I will personally take discoveries of this nature with a good dose of salt. |