|
Honestly, at least when it comes to code in widely used languages, GPT-4 is very good at catching mistakes in generated output after one makes a simple request for a second check. In most cases, there isn't even a need to explain what the specific issue, concern or error in the provided code is. This does make sense as, beyond what has been typed, there is no memory implemented in most of these models, so revisions are currently the only game in town to get more accurate results. What surprised me most concerning this entire situation is that the model did insist on being correct. Normally ChatGPT has been set up to be a bit cautious and more likely to admit to having made a mistake, to the point where if you ask in a direct manner like this lawyer has done, the model may claim to have been incorrect, even when the output was actually correct, in my experience. Bings implementation, of the same underlying model, meanwhile can be so forceful in trying to convince users that the output is correct, even when provided with online resources that show the oposite, that it would not be unreasonable to feel gaslit by that LLM. The rest of this situation was not very surprising and I have do admit, I am happy that this was caught right away. Lawyers actions have a massive impact on countless people every day, if this had not become such a public scandal right away, perhaps a lot of defendants would have suffered under improper representation due to reliance on imperfect models. |