|
|
|
|
|
|
by beemaestro
997 days ago
|
|
|
I have to say, this is pretty neat! Most examples of this that I have seen have been in Python. But the flexibility of REST opens up a lot of possibilities. One thing I’m curious about is if function calling can reduce token usage and by how much. GPT 4 is quite expensive for any large app with a lot of usage. |
|
|
It can definitely reduce token usage significantly, depending on your use case. In the example here https://functionhub.net/docs/examples/new-york-times, it fetches NYT article summaries and does some grouping. You can probably send the html page in your prompt and ask GPT to parse it for you. The html page is about 10K tokens so it would exceed the the 4K and 8K token limits. So offloading the task of parsing this to a function reduces the usage by 80% in the example.
Not all tasks require LLMs. The great thing about function calling is that you can just offload all preprocessing to your functions.