Hacker News new | ask | show | jobs
by sixtyj 406 days ago
In Gemini, it is really good to have large window with 1M tokens. However, around 100,000 it starts to make mistakes and refactor its own code.

Sometimes it is good to start new chat or switch to Claude.

And it really helps to be very precise with wording of specification what you want to achieve. Or repeat it sometimes with some added request lines.

GIGO in reality :)
1 comments
johnisgood 406 days ago
Oh my, I hate it when it rewrites >1k LOC. I have to instruct it to "modify only ..., do not touch the rest" and so forth, but GPT does not listen to this often, Claude does. I dunno about Gemini.
link
diggan 406 days ago
In terms of "does useless refactors I didn't ask for nor improved anything", my own ranked list goes something like: Gemini > Claude > GPT. I don't really experience this at all with various GPT models used via the API, but overall GPTs seems to stick to the system prompt way better than the rest. Clause does OK too, but Gemini is out of control and writes soo much code and does so much you didn't ask for, really acts like a overly eager junior developer.
link
johnisgood 406 days ago
The first time I used Claude, it rewrote >1k LOC without asking for it, but in retrospect, I was "using it wrong". With GPT, even when I told it to not do it, it still did that, but that was some time ago and it was not done via the API, so I dunno. I think I do agree with your list, but I haven't used Gemini that much.

Yeah, they do come across as "overly eager junior devs", good comparison. :D

link
diggan 406 days ago
> With GPT, even when I told it to not do it, it still did that, but that was some time ago and it was not done via the API, so I dunno.

Personally I think it's a lot better via the API than ChatGPT. ChatGPT doesn't let you edit the "system prompt" which is really where you wanna put "how to" instructions, so it really follows them. Instructions put in the user message aren't followed as closely as when you use the system prompt, so probably why it still did something, if you were using ChatGPT.

link
sixtyj 406 days ago
I received this gem in Gemini right now:

I am giving up on providing code, and on checking is it working, because it is very time consuming. Tell me when it starts working. Good luck.

:)

link
tough 406 days ago
I love it when models give up, gives me some hope humans will still be required for the time being lol
link
johnisgood 406 days ago
It is right, it is time consuming. I do not blame it. :D
link