[justrealist]

> hallucinates up a method call that doesn't exist

I actually think it helps to reframe this. It hallucinates up a method call that predictively should exist.

If you're working with boto3, maybe that's not actually practical. But if it's a method within your codebase, it's actually a helpful suggestion! And if you prompt it with the declaration and signature of the new method, very often it will write the new helper method for you!

[lanstin]

If you have a long iterative session by the end it will have forgotten the helpful hallucinations at the beginning, so then phantom methods evolve in their name and details.

I wonder if it is better at some languages than others. I have been using it for Go for a week or two and it’s ok but not awesome. I am also learning how to work with it, so probably will keep at it, but it is clearly a generative model not a thinking being I am working with.

[ExtremisAndy]

No idea about Go, but I was curious how GPT-4 would handle a request to generate C code, so I asked it to help me write a header-only C string processing library with convenience functions like starts_with(), ends_with(), contains(), etc.) I told it every function must only work with String structs defined as:

struct String { char * text; long size; }

...or pointers to them. I then asked it to write tests for the functions it created. Everything... the functions and the tests... worked beautifully. I am not a professional programmer so I mainly use these LLMs for things other than code generation, but the little I've done has left me quite impressed! (Of course, not being a professional programmer no doubt makes me far easier to impress.)

[lanstin]

Interesting. I haven’t tried it with C. Hopefully the training code for C is higher quality than any other language (because bad C kills). Do you have a GitHub with the output?

[nicpottier]

Hah, hadn't thought of this but kind of love that take!