Hacker News new | ask | show | jobs
by crazy_marksman 1009 days ago
Is there a token limit for the train step? Like length of documentation, number of example SQL queries?
1 comments

There isn't technically a limit on the storage side but it's generally better if you keep documentation to a manageable length.

You call vn.train(sql=...) on each individual SQL statement that you have.

What'll happen under the hood is the package will use the 10 most relevant SQL statement examples, 10 most relevant pieces of documentation, and 10 most relevant DDL statements.

If using 10 examples exceeds the (approximate) token limit for the model, it'll pare down to a smaller number that'll fit into the context limit.