[bhntr3]

Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together.

Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be.

But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.

[niklasd]

I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.

[kmill]

I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal.

One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse.

The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.

[derangedHorse]

I've never heard of AI Dungeon before reading your post but even after playing for 2 minutes, I can tell it's going to be huge.

[hnick]

How about full adult? Taking it for a test run and this happened after I told a man to stop copying me. Before this he kept talking about clothes for some reason.

> The man walks away and starts undressing. You shrug and keep following him. Soon, you find yourself naked.

[Reelin]

Library Genesis contains lots (millions?) of fiction ebooks (among other things). It's available in torrent form. Not that I would ever condone piracy or anything.

[gpderetta]

Well now we know what is going to finish asoiaf in case of author existence failure.