[453qtgreq]

The lionisation of GPT technology (which is nothing special, apart from the scale of execution) is quite embarrassing. Have we forgotten that each of us has, within our own heads, a computer vastly superior to this toy (and one that needed $12 million in compute costs alone to train, while mine at least runs on porridge and cheese sandwiches)?

Just for fun, try a prompt like "Write a new letter" (which eventually gave an error). Or "Write something new" (I was then told I was "making too many requests").

[dwaltrip]

You are thinking too narrowly.

If you asked me, a human, “tell me something new” without any additional context, I would reply with the human equivalent of “error, request is overly vague” as well.

[453qtgreq]

On the contrary. My example is to show that ChatGPT does fairly well with constrained questions and textual forms, and gets worse with broader questions.

"What's new with you" is a relatively common greeting.

[ben_w]

First, “Write something new” is very different from “What’s new with you”. In the case of the GPT family LLMs, there fundamentally isn’t anything new with it the way there is for us.

Second, to your previous point, you’re mixing up training with inference. The training cost is $12 million, which you should compare to your total cost of being alive and educated from conception to whatever age you think is comparable to it’s abilities. The inference cost is much smaller; while I don’t have figures for the cost per query on ChatGPT, I have calculated an upper bound for Stable Diffusion, which is 3150 Joules per image (on my hardware), which makes that AI not only cheaper than the cost of an equivalent human artist, but also (assuming $0.1/kWh) cheaper than the cost of getting someone at the UN abject poverty threshold to enter the prompt.

Third, even though it’s got enough errors for domain experts to spot and be annoyed by, it’s quite often at the level of being interestingly wrong, like a university student. Except, unlike humans, it’s equally able to answer kinda-OK-that’s-interesting about JavaScript HTTP requests in French, how to make a New York cheesecake in German, to summarise utilitarianism in Chinese, the Declaration of Independence in Esperanto, the history of the British Empire in Welsh, to give the Einstein Field Equations in LaTeX with description of what each term means, to give a plausible reason when asked «Why do Germans say “Deutsche Sprache, schwere Sprache”?», to respond to the prompt “Make a joke about klein bottles” with “Why did the mathematician wear a Klein bottle to the party? Because he didn't want to be a one-sided personality!” (which I can’t find on Google and I want to cautiously suggest may actually be a novel joke), and when given the prompt “Describe a smartphone using the tropes and style of HP Lovecraft”, it gave me this, which I think is wonderful:

> A smartphone, with its sleek obsidian exterior and glowing, unearthly screen, exudes an aura of otherworldly power. Its many tentacles, in the form of slender cords and wires, writhe and twist as if alive, connecting it to the vast, unseen network of the internet. With a mere touch of its inky black surface, one can conjure forth all manner of arcane information and eldritch knowledge, summoning forth eldritch horrors and forgotten deities alike. The very touch of the smartphone's cold, clammy surface is enough to send shudders of fear and revulsion down the spine of even the most stout-hearted individual. Its powers are not to be trifled with, for to gaze upon the smartphone's eldritch screen is to risk madness and insanity.

Sure, you can still trip it up (even ignoring server errors and the deliberate guard rails about dangerous, illegal, or controversial subjects), as I also tried the trick question “When did Neil Armstrong first walk on the surface of the Earth?” and got the obvious wrong answer of 1969 rather than 1930-ish. But humans can also be tripped up with trick questions, as demonstrated by, e.g. the BBC comedy quiz show QI.