The thing that annoys me is that you can ask it to do something specific but it does what it wants.
For example I had a massive sql query that was loads of statements unioned together, I said “for each statement remove this filter and add this filter” and it would go “certainly, here are the first four, I have used an example table name feel free to change it” then I’d say “can you do it for all the statements, not just the first four” and it would say “of course here you go” and just give me the first four but also makes them useless by changing the table name!
I’ve got great hopes that one day I can get it to help me shape the code in a way that the jetbrains ide’s can’t today - for those I have to choose from a set of available operations - I want to talk to it and get it to change the code in a set of operations that I choose!
Perhaps, but I wouldn't mind if the models just answered "I'm sorry but I don't have an answer to your question at the time". In fact I think that would be a great answer that would increase the amount of trust I have in ChatGPT.
Instead the model decides to make stuff up and pretend that it knows. That's vastly worse.
It reminds me of the early days of DuckDuckGo, when if you searched for something obscure with no matches online it would still fuzzy match some garbage like a binary blob in a Chinese PDF while Google helpfully would just tell you that it couldn't find anything.
> Instead the model decides to make stuff up and pretend that it knows. That's vastly worse.
Does the model know it doesn’t know though? Does “know” even make sense as a concept here? I don’t know if it can really introspect like that, but of course it would be so much better if it could can have some sort of confidence score with each answer.
For example I had a massive sql query that was loads of statements unioned together, I said “for each statement remove this filter and add this filter” and it would go “certainly, here are the first four, I have used an example table name feel free to change it” then I’d say “can you do it for all the statements, not just the first four” and it would say “of course here you go” and just give me the first four but also makes them useless by changing the table name!
I’ve got great hopes that one day I can get it to help me shape the code in a way that the jetbrains ide’s can’t today - for those I have to choose from a set of available operations - I want to talk to it and get it to change the code in a set of operations that I choose!
One day maybe :)