Hacker News new | ask | show | jobs
by Oras 712 days ago
Word of caution when using perplexity to summarize URLs. I have tried the API and had lots of hallucinations.

It worked better in the UI, I only had it with the API.
2 comments
freilanzer 712 days ago
I've been evaluating OpenAI (GPT-3.5 and -4o) for a project at my company and I have seen differences in quality between the api and the web version as well. Same prompts. The api version seems to vary a lot more in quality.
link
egorfine 712 days ago
Play around with temperature. Summarization is quite a defined task, so lower the temp a lot.
link
freilanzer 712 days ago
It's not just that. Following instructions, text generation, etc. all seem to be worse with the api version. At least, when it comes to 3.5 - 4o is much better. Ironically, 4o had problems with correct json output where 3.5 hadn't. It produced ```json {...}``` instead of just the json object.
link
cpursley 712 days ago
Yeah, I don’t know why their API is inferior to the UI. Pretty disappointing. I’ve had better luck with my own “clone” that makes a web search and summarizes the results into an answer.
link