[al_borland]

The knowledge isn't of any use to us unless it is understandable to us. Many species understand things about the world around us that we are unable to explain or understand, even if it's just pure instinct on their part. These things are very useful to them, but have no value to us until we can understand and explain it, which then allows us to make use of it.

People saw birds fly for all of human history, but it was only recently that humans were able to make something fly and understand why. Once we understood, we were able to do amazing things, but before that, the millions of birds able to fly were of no help beyond inspiration for the dream.

[miki123211]

This is not true.

We use drug-sniffing and guide dogs in a way similar to how we use LLMs. We don't really understand them at a fundamental level, we can't make electronic dog noses (otherwise we'd dispense with the silliness and just install drug detectors instead), but dogs are useful, so we use them.

[al_borland]

We don’t blindly trust the drug-sniffing dog. The dog gives a signal that it was trained to give, then humans understand what that signal means and verify the accuracy. Without the human understanding in the loop, the dog’s ability is of little value.

Without a human in the loop and LLM could churn away spitting out results, some right, some wrong, and it would be of no consequence. Not much different than wild dogs sniffing each other.

[vuquangchien]

The knowledge isn't of any use to us unless it is understandable to us => it seems that you have shifted the goalpost here. In the dog example, humans still don't understand how dogs sniff, but it is of use to us and thus is meaningful. The same for quantum effects - we don't understand how it works. We just guess that it works reliably and make use of it.