These llms are weak facsimiles of the brain, not humans. The real world is the ultimate training model, it can't be fully substituted with a bunch of strings.
This sounds like the old question "if a blind-from-birth man (who can recognize squares by feel) gained sight would he be able to recognize squares visually?"
And since that question has been answered in the negative, I'm inclined to agree.
And since that question has been answered in the negative, I'm inclined to agree.