[antibasilisk]

Between GPT-3 and GPT-4 the precision required for prompts was decreased significantly. In theory it should reach the point where a project manager type person would be able to describe what is needed and it would simply do it, the main thing missing from attaining that is that GPT-4 basically never responds to questions with requests for clarification, otherwise, a whole team of developers could be reduced to just one proofreader.

[xwdv]

That’s not very impressive. With very little precision I can throw a few keywords into google and get a full answer and perhaps some code snippets from stack overflow to solve whatever problem I have in the moment.

Except you can’t just blindly copy code snippets, you have to read the author’s explanation and perhaps adapt things to your own code sometimes, or reject their solution entirely. GPT-4 can’t do this because it doesn’t actually know what the hell it’s doing, it’s just putting stuff together in a form that is most probably correct based on what it has seen in training data for past examples.

I fear for the layman who sees a bunch of AI generated code and think it must be right. Who knows what bugs, security flaws, or performance issues they will run into, that they have no idea how to solve or even to begin asking a prompt for.

[UncleEntity]

> That’s not very impressive.

That’s kind of impressive.

I usually have zero luck with stack overflow except for super trivial things.

Some of the stuff I’ve done has no documentation and I had to find an example (like one other person in the entire history of mankind though this was a good idea) on some random repo on GitHub. Or I’m implementing code from a paper written 30 or 40 years ago and there’s no example code to look at. I could ask on stack overflow but who needs that abuse?

Admittedly I just do this to amuse myself and think that having a LLM digest a paper and spit out code is the bee’s knees.

[ozfive]

> Admittedly I just do this to amuse myself and think that having a LLM digest a paper and spit out code is the bee’s knees.

This is what I have been doing also. Verification/Implementation of Software Engineering papers will be juicy especially with image ingestion.

[osigurdson]

You must not have used GPT very much based on this response. It is pretty great.

[zenlikethat]

The thing that makes ChatGPT (especially 4) great though is that everything is hyper customized to you and, given a little prompt context (which is pretty trivial to automate and will be built right in to our tools one way or another), it can produce working code that fits your exact needs extremely rapidly.

e.g., if I type into google, "Build me a React component todo list that loads its data using fetch() to /api/v1/todos and is styled with Tailwind", I'm going to get a bunch of stuff that I can maybe use, if I invest 15 minutes wading through cancer-inducing blogs. Whereas at least for trivial problems, ChatGPT just tesseracts something that's extremely close and you copy-paste it in, change a few things, boom, component done. Something doesn't work? Often a follow up message fixes it.