It’s true across the board. How to get good results with coding using AI? Be a good coder. How to get good financial advice? Be financially literate. How to get good medical advice? Be a doctor…
Makes ya wonder: where is the intelligence coming from?
I would argue coding is different from those other categories. I think you can ask AI expert level questions without being an expert in many fields if you are smart and well educated in general.
> “We were somewhat surprised by how good the advice was,” Choukhmane said. “Especially when you read the kind of questions people asked, it was not a given that the advice would line up with what academics think are good financial principles.”
TFA goes on to point out that more academic prompts did better still -- but a major point was that, even with naive and simple questions, the advice was still surprisingly good.
And similarly, quoting from the article which TFA cites:
> First, following LLM advice would move most survey respondents closer to the prescriptions of life cycle theory relative to their current behavior, including broader participation in diversified equity funds, equity shares that decline with age, and sizeable saving buffers. Second, replacing individual-written prompts with academic prompts moves LLM advice even closer to life cycle theory, with better consumption smoothingand less reliance on simple heuristics.
To me its felt like the LLM sort mirrors what you say to it and how you say it, to the point where I now include additional instructions to steer the model "Never mirror the user’s present diction, mood, or affect". It feels like I get a better experience but to be honest i haven't really done any benchmarking especially on newer models
Makes ya wonder: where is the intelligence coming from?