Hacker News new | ask | show | jobs
by myrmidon 1177 days ago
> most of those sentences are meaningless so they won't come up in normal use

Feel free to come up with a better entropy model then. Stackoverflow gives me confidence that it will be between 5 and 11 bits per word anyway [https://linguistics.stackexchange.com/questions/8480/what-is...].

> if statements can grab patterns just fine in most languages, they're not limited to pure equality

This does not help you one bit. If you want to produce 9 sentences of output per query then regular expressions, pattern matching or even general intelligence inside your if statements will NOT be able to save the concept.
1 comments
ben_w 1176 days ago
> What is the entropy per word of random yet grammatical text?

More colourless green dreams sleep furiously in garden path sentences than I have

> This does not help you one bit.

Dunno, how many bits does ELIZA? I assume more than 1…

link
myrmidon 1176 days ago
> What is the entropy per word of random yet grammatical text?

That is what these 5-11bit estimates are about. Those would correspond to a choice out of 32 to 2048 options (per word), which is much less than there are words in english (active vocabulary for a native speaker should be somewhere around 10000-ish).

Just consider the XKCD "thing explainer" which limits itself to a 1k word vocabulary and is very obviously not idiomatic.

If you want your big if to produce credible output, there is simply no way around the entropy bounds in input and desired output, and those bounds render the concept absolutely infeasible even for I/O lengths of just a few sentences.

Eliza is not comparable to GPT because it does not even hold up to very superficial scrutiny; its not really capable of even pretending to intelligently exchange information with the user, it just relies on some psychological tricks to somewhat keep a "conversation" going...

link
ben_w 1176 days ago
> Eliza is not comparable to GPT because it does not even hold up to very superficial scrutiny; its not really capable of even pretending to intelligently exchange information with the user, it just relies on some psychological tricks to somewhat keep a "conversation" going...

That's kinda the point I was making — tricks can get you a long way.

The comparison with GPT is not "and therefore GPT is bad" but rather "it's not necessarily as smart as it feels".

Perhaps I should've gone for "clever Hans" or "why do horoscopes convince people"?

link