IF it could (I wouldn't know one way or the other), I'd consider that a damning indictment of the Bar Exam failing to test for sentience, rather than evidence of GPT-4 having attained the same.
Bar exam is not a test of sentience but of the ability to recall, interpret, and apply the law. Because law is an entirely textual thing, I would expect GPT to be exceedingly well suited for it.
I've said for a long time that most doctors and lawyers are just databases with quick and imperfect retrieval.
Maybe a useful way to think about it is that we don't know for sure what intelligence is nor how the gradient of intelligence expresses itself. For instance, does a human grow in intelligence as it ages (a baby doesn't "know" stuff, but has the capacity for learning and then applying in new situations as its experience grows).
I interpret your statement as implying that ChatGPT is somewhere on the spectrum of intelligence, yes?
Maybe the "talk about your issue and get a diagnosis" area of practice (internal medicine?); since far less sophisticated manual labor can't yet be automated, surgeons are going to be irreplaceable for longer than, say, BI, and many backend or frontend developers.
I wonder if people could be more honest with a sub-sentient AI than they could be with a real life doctor. I bet they currently are more honest in their Google searches than they are with the doctor.
Right. Would we be impressed if a layman could pass the bar, given infinite time and access to the entire Internet (including the copious amount of bar exam study guides and worked example problems)? If not why are we impressed that a language model trained on that data can?
Meanwhile when I ask ChatGPT which of six numbers are odd, it confidently reports a mix of even numbers, odd numbers, and letters.
This is a fun milestone but the angst above about the “end of commoditized intelligence” etc. is unwarranted.
My guess is that AI omitted 'a' because this is essentially how natural language processing works. Perhaps it cannot see 'a' because the input has been stripped of 'a' or 'the', and so on.
I've said for a long time that most doctors and lawyers are just databases with quick and imperfect retrieval.