Hacker News new | ask | show | jobs
by keyme 1242 days ago
Chat gpt will never get smarter though (learn anything new), unless actual people use stack overflow to provide the training data.

The most important comment on SO is the one where the OP says "I did this and that, and can confirm it's working". There is no such data when you only use the chat bot. It's a major problem. Perhaps the solution is a more tailored "stack gpt", where the "conversations" are published along with the human responses.

3 comments

ChatGPT is already “smarter”. I can ask ChatGPT for the answer to my very specific problem and it can answer it snd give me code.

For instance, I needed to write a simple script that gave me all of the roles containing a list of policies where you can specify multiple policies from the command line using “-p” multiple times.

If I look it up on Google, I get this response which is close enough

https://stackoverflow.com/questions/66127551/list-of-all-rol...

And I still have to make slight modifications.

Of course I could write the entire thing myself just by looking at the boto 3 docs.

But instead I told ChatGPT

“ Write a Python script that returns a comma separated list of arns of all AWS roles that contain policies I specify with the “-p” parameter using argparse”

Then I told it, “that won’t work with more than 50 roles”. It then corrected the script and used a paginator.

Yes, I had to know enough to recognize the bug. But I count have written or modified a Python script that fast and I write boto3 based Python scripts all of the time.

I definitely could have told it that the first version wasn’t returning all of the results and it would have corrected itself like it did and added a paginator

They know this is a problem, so they’re hiring contractors to write code and label it to teach the AI.
I think the thumbs up and down button it shows on each response can be good for that. I think it can already classify similar questions so if it consistently gets a thumbs up or thumbs down it can become smarter too.