[importantbrian]

We might eventually get a good LLM to SQL tool, but my experience with them is that they make slick sales demos, but are worse than useless in the real world.

You have to know SQL to use them. They produce a lot of code that looks correct and produces correct-looking results with subtle errors. So you can't just hand it to someone who doesn't know SQL and let them query the database, but that's the use case where something like this would be valuable. You have to be experienced with SQL and know all the peccadillos of the DB you're working with to check the query and output for correctness.

For someone like me who is experienced with SQL, I can write simple queries just as fast as I can figure out how to prompt the LLM to get what I want. Where a tool like this would be really helpful is if it could help me write more complex queries more quickly. However, it is non-trivial to get the LLM to generate complex queries that take into account all the idiosyncrasies of your specific data model. So again it ends up being much faster for me to just write the query myself and not involve the LLM.

Where I think LLMs go wrong with SQL is that to write good SQL you have to have a deep knowledge of the underlying data model, and the LLMs aren't good at that yet.

[apoorvnandan]

Strongly agree. LLMs need something more than just the DDL of the tables and an instruction to write useful SQL in the real world. However, I've had decent success by (1) integrating heavily with a semantic layer on top of your database, and (2) going the agent approach where the LLM is allowed to run different queries and explore the data before writing the final query.

[staticautomatic]

Asking the LLM if the query can be optimized a few times and then checking it in a planner works surprisingly well for me but YMMV.