[bsenftner]

There is a subtle aspect of LLM AIs that is lost to most people: they are trained on the entirety of the Internet. That means whatever topic you ask these LLM AIs, there are multiple instances of that same information with different levels of seriousness and accuracy in their treatment of the subject.

For example: if one asks a question using street slang, the answer generated will be generated from training data about your subject, but from online sources that used street slang in their conversation about that issue. Likewise, if you use ordinary language for your question, the generated response will be from ordinary language conversations of your topic. However, if your question concerns any type of formalized knowledge, by asking your question using the formal language of experts in that topic, then the generated AI answer will come from training data that used this same formal expert terms, and are most likely to be correct, because they come from discussions of that subject’s matter experts.

Plus, don't use LLMs for fact retrieval, use them as strategy guides. They really excel as strategy advisors.

[choilive]

Theres actually even more subtlety here, in all of your examples the "knowledge" should theoretically be embedded nearby each other in the same vector space, so regardless of the style of language used, semantically they should all pull from similar weights, and thus give similar answers. This is one of the reasons why LLMs are so powerful.. because they seemingly understand the semantic relationships of words so regardless if the prompt is posed casually or formally it should give similar answers in terms of factuality. I agree with you that LLMs today should be primarily used for more creative output.

[bsenftner]

That assumes that street slang discussions, using entirely different conceptualizations of ideas, would indeed be embedded nearby one another. Plus, both the street slang and ordinary language will tend to treat the information in a less precise, a less concept discriminating manner (meaning the subtle distinctions between issues may be lost in their discussions). In my tests, I find one indeed needs to use the subject matter expert for precise treatment of formal knowledge and generated answers that are more accurate.