[rockemsockem]

I find the Copilot completions to be slightly better than IDE auto complete, but the latency is too high for it to be useful IMO. It's to big a break in concentration to wait for it to type almost what you were about to.

IMO chat-based question-asking when encountering unfamiliar language constructs or framework idioms is where LLMs shine in coding assistance.

[packetlost]

> I find the Copilot completions to be slightly better than IDE auto complete, but the latency is too high for it to be useful IMO. It's to big a break in concentration to wait for it to type almost what you were about to.

This matches my experience perfectly. It gets decent/usable results like 40% of the time, but latency kills it.

> IMO chat-based question-asking when encountering unfamiliar language constructs or framework idioms is where LLMs shine in coding assistance.

I have never found this to be productive. It gets details wrong frequently enough that I can't trust the output which effectively doubles (or more) the cognitive effort because you have to validate that it's correct against a more reliable source, or find out the hard way that it's wrong when code/systems break.

[rockemsockem]

Interesting, what systems are you asking questions to? I find GPT-4 is pretty much always correct about simple questions I ask. I just avoid using it for anything really deep and refer to documentation instead.

[packetlost]

I've mostly used GPT-4 and Gemini Pro. For simple stuff, it can be correct, but it highly depends on the topic. I've found it very effective as a dictionary or thesaurus but for technical questions, even simple ones like small help with a Scheme macro, it's near useless. I imagine if I was asking it about, say, Python/Django APIs or basic HTML/webdev stuff, it would do better.

[rockemsockem]

When you say technical questions do you mean writing code, explaining code, or something else? If it's writing code I agree, I don't often ask it to explain code unless I want to know about some obscure syntax, but it works pretty well for that. I would not trust it to explain complex logic in a function.

[packetlost]

I've tried all of the above. I've asked it to summarize information about what languages in a class had strong symmetric multiprocessing characteristics and it made up over half of the details. I've asked it to generate code for a simple linked list and it created garbage. I've asked it to summarize what code does and it gets it blatantly wrong.

These are just a few examples. I've found it to be nearly useless for anything where a semblance of correctness matters. Every single thing it says must be validated against external sources and it can easily lead you down rabbit holes.

GPT-4 was a little better than 3(.5), but I've yet to see it be good at anything enough to warrant the price tag or the time.