> There’s literally nothing an llm can write or tell you that you can’t write yourself or find in a manual somewhere.
That's like saying, there's literally nothing a service business can do for you that you can't do yourself. It's only true in a theoretical sense, if neither time nor resources are a constraint.
In such hypothetical universe, you don't need a dentist - you only need to spend 5+ years in medical school + whatever extra it takes to become proficient with tools dentists use + whatever money it takes to buy that equipment. You also don't need accountants, lawyers, hairdressers, or construction companies. You can always learn this stuff and do it yourself better!
Truth is, time and attention is finite. Meanwhile, SOTA LLMs are cheap as dirt, they can do pretty much anything that involves text, and do it at the level of a mediocre specialist - i.e. they're literally better than you at anything except the few things you happen to be experienced in. Not perfect, by no means error-free - just better than you. I feel this still hasn't sunk in for most people.
> No, it’s not. You’re making my statement abstract for the sake of arguing.
> I’m not a cook, doctor, or a lawyer. I can’t prepare meals for a party of more than 2.
They are demonstrating how over-broad your own statement was with an *equivalent* statement to show how it only passes on an unhelpful technicality.
Immediately after your quotation is this:
> you only need to spend 5+ years in medical school + whatever extra it takes to become proficient
LLMs pass the bar exam and the medical exam. These are things which I assume I would be able to do myself if only I were willing to dedicate 5 years of my life to each.
> I can write exactly all code an llm could write.
I can often see many errors in the code that ChatGPT produces. Within my domain, it's just a speed-up, a first draft I have to fix. Outside my domain, it knows what I *can't* Google because I've never heard the keyword that would allow me to.
On legal questions, ChatGPT (despite passing the bar exam) seems to make up cases. I belive this because I can google the cases and fail to find them. Is this because they don't exist, or because they're not indexed on Google? I don't have the legal background necessary to know — and it would take me years to get the knowledge necessary to differentiate "it's worse than first glance" from "it's better than second glance".
Even if LLMs were "a memorization machine" (they're bad at that), the statement is obviously false because actutal literal memorisation machines (books, video recorders, Google) cannot pass these exams.
LLMs only started to because they could follow the questions.
But even that aside, it doesn't matter why LLMs can do what they can do or what else can also do that, what matters is that it would take most humans several years to get to the level of current LLMs in a subject that human isn't already familiar with.
I don't know how you compare, but ChatGPT's English grammar and vocabulary are significantly better than mine. And when I prompt it appropriately, it also seems to be a better creative writer than I am, at least for short pieces.
> Chatgpt (and other LLMs) are awful at creative prose.
As are most humans.
Don't get me wrong, what I've seen from even the better LLMs have a certain voice and tropes and sacherine worldview that isn't dark enough where it needs to be for the story to work; but on the other hand, what I see on some fiction writing subreddits… the AI is often a genuine improvement over amateur writers, even in cases where the AI contradicts itself about plot elements.
Which is frustrating, because I have the feeling the novel I've been trying to finish writing for the last decade may be usurped by AI before I get my final draft.
> what I see on some fiction writing subreddits… the AI is often a genuine improvement over amateur writers
What point are you trying to make here? That amateur writers are amateurs?
That AI is only "often" an improvement over an amateur?
> Which is frustrating, because I have the feeling the novel I've been trying to finish writing for the last decade may be usurped by AI before I get my final draft.
This statement shows such a warped attitude towards art and the creative process. What do you mean "usurped?" Do you actually believe that LLMs will overtake humans when it comes to creative works?
If so, you don't really understand what is compelling about the written word or what makes for good writing and reading and it's no wonder you feel as though your own writing is so substandard.
I highly doubt your writing is that bad. Especially if you've been working on it for a decade.
It's giving a reply that "sounds right to a human" in context. Which, on small scale, is exactly what they, me and you are also doing when writing (or speaking), except for the infrequent cases when we force ourselves to reason through stuff very. slowly.
(This is why I believe LLM performance is best judged against human inner voice/system 1 reasoning, not the entirety of human thinking. When thinking with system 1, people don't really have an idea what they're doing either - they're just doing stuff that feels right.)
Also note that "sounds right to a human" is literally the loss function on which LLMs are trained, so between heaps of training inputs and subsequent extensive RLHF, the process is by its very construction aiming optimizing for above-human-average performance across the board.
Even documented libraries can be a struggle, especially if they are not particularly popular. I'm doing a project with WiFi/LoRa/MQTT on an ESP32. The WiFi code was fairly decent, but the MQTT and especially LoRa library code was nearly useless.
Sonnet 3.5 fails to generate basic JetpackCompose libraries properties properly. Maybe if somebody tried really hard to scrape all the documentation and force feed it, then it could work. But i don't if there are examples of this.
Like general LLM, but with complete Android/Kotlin pushed into it to fix the synapses.
Of course, why wouldn't it? It's a generative model, not a lookup table. Show it the library headers, and it'll give you decent results.
Obviously, if the library or code using it weren't part of the training data, and you don't supply either in the context of your request, then it won't generate valid code for it. But that's not LLM's fault.
Even if you absolutely have to use an LLM for some reason, there are already perfectly good LLMs for code generation that you can comfortably run on commodity hardware.
That's like saying, there's literally nothing a service business can do for you that you can't do yourself. It's only true in a theoretical sense, if neither time nor resources are a constraint.
In such hypothetical universe, you don't need a dentist - you only need to spend 5+ years in medical school + whatever extra it takes to become proficient with tools dentists use + whatever money it takes to buy that equipment. You also don't need accountants, lawyers, hairdressers, or construction companies. You can always learn this stuff and do it yourself better!
Truth is, time and attention is finite. Meanwhile, SOTA LLMs are cheap as dirt, they can do pretty much anything that involves text, and do it at the level of a mediocre specialist - i.e. they're literally better than you at anything except the few things you happen to be experienced in. Not perfect, by no means error-free - just better than you. I feel this still hasn't sunk in for most people.