[krishadi]

The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just calling an API.

The dust around language models needs to settle a bit, for a useful framework to emerge from it.

For our own use-cases, I built a framework from scratch, and it was the best decision we made.

[data_maan]

> For our own use-cases, I built a framework from scratch, and it was the best decision we made.

My thinking precisely. So you just used the "raw" OpenAI (I presume?) API, and no other tech on top?

[krishadi]

Exactly. The most important part was working with Jinja templating. So, openai + jinja2.

[riter]

very much agreed re: dust settling.

it makes no sense deploying any of these libraries to prod. as-is. best to understand a configuration / workflow / tuning / etc. that fits your data best and write it from scratch in golang/rust/whatever.

[Dowwie]

Are these computationally expensive operations? If not, Elixir could fit.

[krishadi]

They are not all computationally expensive. The rate limiting step here is the LLM call itself over the API. So, async is definitely needed. The other aspects would be loading the template from filesystem. I would assume this could be something that's needs to be optimized in the application.