Hacker News new | ask | show | jobs
by jasfi 1139 days ago
They really need to release an official API. I was trying to find out the pricing for API access, but it's not available yet. The API used by this Python package seems to go through an unofficial route.
3 comments

I mean, this "API" is free whereas an official API could be charged

I guess it's only fair that we may scrape Bard just like Bard scrapes us? I think it's called "disruption" :-)

They have released it I believe. I can see Palm being available: https://cloud.google.com/model-garden
Thanks for sharing the link. It is a testament to Google's failure marketing its cloud offerings that, despite wanting to try out this API, I had been unable to find it until you helpfully shared the link.

I went down the cloud console rabbit hole to play with it. Only the bison model was available (of the non-descriptively named gecko, otter, bison, unicorn models). For innocuous prompts I got this error: "The response is blocked because the input or response potentially violates our policies. Try rephrasing the prompt or adjusting the parameter settings. Currently, only English is supported."

Where it did respond, I liked the quality. It was comparable to the GPT-3.5 API. Perhaps a little better even.

From personal experience, Bard's quality seems to be between GPT-3.5 and GPT-4, closer to GPT-4 if you have to bet. Except when fresh or live data matters, where Bard is clearly superior. (Bard's training data is up to Feb 2023 compared to ChatGPT's Sep 2021, and Bard also gets live data from Google search.)

MMLU benchmark score agrees with this estimate: GPT-3.5 (70.0%), PaLM 2 (81.2%), GPT-4 (86.4%).

Does this take into account failure to consider context? I have a lot of trouble with bard not using context from previous messages.
Bard's context length is equal to that of GPT-3.5.
Thanks, it didn't show up immediately when I searched for it.

But it looks like you have to rent an instance of Vertex AI to use it. It doesn't seem to be pay per use as OpenAIs model are priced with.

Try signing up on the waitlist and ping a sales rep at GCP for early access - https://developers.generativeai.google/products/palm

Worked for us.