| Sorry but I think we have very different perspectives here. I assume you mean that LLMs can generate new insights in the sense of producing plausible results from new data or in the sense of producing plausible but previously unknown results from old data. Both these things are definitely possible, but they are not necessarily (and in fact often not) good science. Insights in science are not rare. There are trillions of plausible insights, and all can be backed by data. The real problem is the reverse: Finding a meaningful and useful finding in a sea of billion other ones. LLMs learn from past data, and that means they will have more support for "boring", i.e. conventional hypotheses, which have precedent in training material. So I assume that while they can come up with novel hypotheses and results, these results will probably tend to conform to a (statistically defined) paradigm of past findings. When they produce novel hypotheses or findings, it is unlikely that they will create genuinely meaningful AND true insights. Because if you randomly generate new ideas, almost all of them are wrong (see the papers I linked). So in essence, LLMs should have a hard time doing real science, because real science is the complex task of finding unlikely, true, and interesting things. |
I’d be surprised if you have used LLMs beyond the classic chat based linear interface that is commonly used and still have the opinions you do.
In my opinion, once you combine RAG and agent frameworks with raw observational input data they can absolutely do real reasoning, analysis, and create new insights that are meaningful and will be considered genuine new science. This project/group we are discussing have practically proven this with their replication examples. The reason this is possible is because the LLM is not just taught how to repeat information but it can actually reason and analyze at a human level and beyond when utilizing it’s capabilities within a well designed cognitive architecture using agents.