[verdverm]

> the trade-off is that instead of returning links it returns a generated soup of words that are semantically close to your "query".

I get links in my responses from Gemini. I would also not describe the response as soup, the answers are often quite specific and in the terms of my prompt instead of the inputs (developer queries are a prime example)

[quectophoton]

I call them a "soup" because AFAIK there's no intent behind them:

I'll stop calling them a soup when the part that generates a human-readable response is completely separate from the knowledge/information part; when an untrained program can respond with "I don't know" due to deliberate (/debuggable) mapping of lack of data to a minimal subset of language rules and words that are encoded in the program, rather than having "I don't know" be a series of tokens generated from the training data.

[verdverm]

Those are called Agents and already exists today. I've been prompted for more information when the agent realized it didn't have all the context it needs

[quectophoton]

Don't agents still depend on LLMs to produce a human-readable response, rather than as a source of information/knowledge? And aren't they still vulnerable to prompt injection attacks, due to being unable to separate the information/knowledge part vs the prompt, because their prompt "parsing" is coupled to an LLM?

[verdverm]

If you give them a fair and reasonable go, you'll discover more than asking leading questions on HN. In example, there are many things you are unaware of as possibilities, like how easy it is to undo code changes to the last checkpoint (copilots chat checkpoint, not git or vcs). They can also make use of all the external tools, knowledge repositories, and search engines we use.

My personal experience has led me to increase my monthly spend, because the ROI is there, the UX is much improved

Hallucinations will never go away, but I put them in the same category as clicking search results to outdated or completely wrong blog posts. There's a back button

[quectophoton]

Yeah that has been on my backlog. I admit that I haven't given them too much priority, but at some point I want to try an AI agent that works offline and is sandboxed.

[verdverm]

The frontier models like Gemini are so much better than the open weight models you can run at home, night and day difference. I have yet to try the larger open models on H100s

I'm keen to build an agent from scratch with copilot extension being open source and tools like BentoML that can help me build out the agentic workflows that can scale on a beefy H100 machine