Hacker News new | ask | show | jobs
by aiappreciator 1210 days ago
At very minimum, better novels.

Current text transformers are horrendous in writing long form stories (ie, longer than 1 page).

Because they don't have a concept of long-term memory. It has to keep everything in its short term memory (the context window), which is at most 2k words right now. Everything else is discarded, so the AI is unable to keep track of past events.

This AI probably tries to summarise past events into short summaries. Sort of like how humans don't remember details of past events (What did you eat last week), only tracking important or unusual events. This helps massively optimize the memory of the AI

Novels are probably the grand challenge in text-AIs, because they require multiple things.

1. Long term memory

2. Multi-party state tracking (What happened to whom, how is relationship graph between multiple characters changing, what is happening in the background, or the world, despite not being mentioned in the text explicitly)

3. Multi-party theory of mind (The AI must infer the internal mental state of characters despite not being explicit in text)

4. Accurate understanding of human motivations/desires, which are the driving force behind stories.

As such, AIs that can write long fictional stories is also capable of: 1. Deception (Plot twist/surprises) 2. Emotional manipulation (Pulling your heart strings) 3. Long term planning (The simulated characters need to plan long term, with an effect on the world-state)

Needless to say, it will be extremely dangerous. But that AI will also master therapy, sales, supervising children, customer service etc, as it now has an strong understanding of human behaviour.

Still, all of that is quite a few years away. In the meantime, AIs that can assist human fiction writers is very possible, humans do the long term tracking and comprehension, the AI can help fill in dialogue, polish up writing styles, describe scenery or objects etc.

Novel writers are a great testing ground despite limited economic value, because novel writing AIs are risk-free and error-tolerant. Novel writers are generally also extremely excited about AIs, unlike artists.

1 comments

Why would we want an AI that writes novels though? Is this a "to see if we can" thing?

Let's say this or some future AI system writes better novels than any human author at a fraction of the cost. Novel writing is solved.

What will we have achieved?

I wish I could opt out of this world you want to create, where if you achieve your vision, I will be utterly useless and obsolete.

As I said: "that AI will also master therapy, sales, supervising children, customer service etc, as it now has an strong understanding of human behaviour."

Novel writing is like the training ground for emotional intelligence in AIs. Fiction writing itself is not economically important, but the skills learnt from it is.

As a passionate hobby writer myself. I say you should actually go and try the writing tools, sudowrite/verb.ai are examples. Once you start using them, you start realizing how much they suck at writing stories, and your worries will go away. Unlike art AI, fiction writing AI has not really improved significantly. The core challenges are unsolved since the days of GPT-2.

There's also little research money coming in. Having a truthful, helpful, inoffensive AI is the polar opposite of what you want in a story AI, which should be deceitful, aggressive, and offensive.

I tried pasting some paragraphs from my work in progress novel into ChatGPT, and asked it to "improve the writing". The output was exactly the kind of crap you'd expect from someone well read but with no writing experience. E.g. way too many adjectives, and reading as if it had been written by looking up every other word in a thesaurus. I tried having it add some paragraphs, and similar thing: A whole lot of beginner-level writing of the "first he did X, and then he did Y. A bit later he did Z" type of writing. And that's testing just the very superficial language issues and ignoring plot entirely

It's impressive it can even do that, and it'll improve, but anyone looking to these tools to generate good fiction at this point will be disappointed.

>> Why would we want an AI that writes novels though?

Well, if it was "an AI" like Lt Cdr Data, then we would want it to be able to write novels, among other things, just because humans can, and we presumably want to create artificial humans, no?

Maybe not, I think it's a very bad idea to create artificial humans. But the systems we're talking about are not artificial humans, they're the kind of system that everyone on the net has started calling "an AI" in the last few months (I know because it bugs me no end when people do that, but now it's everywhere so I can suck it up, it won't change). Those are only "AIs" in the very specific sense that everyone calls them "AIs", and not because of any of their real capabilities.

And the point is that those "AIs" that we have right now are not capable of writing novels. They are capable though of producing lots and lots of spam spam Spam SPAM.

And they will. There's already so many novels, short stories, novellas, novelletes, flash fiction stories etc etc written by humans, that a human lifetime is not enough to read them all. What is the "AI" going to add to all that? Another human lifetime's worth of spam?

Maybe that's not such a big problem. If I already couldn't read all the books written by humans up 'till now, then I can spend the rest of my life reading only books written by humans, simply by checking the publication date and rejecting any book written after the creation of book-writing "AI" (which we don't have yet).

I'm trying to say, we can avoid reading spam, nowadays, it mostly just clutters our inboxes. We can avoid reading "AI" spam, and it doesn't matter if it will get bigger and bigger or not.

Maybe the future web will be divided into a wastebasket for "AI" spam, and the rest. That's a bit of a bummer, but the web is already divided into shit (99%) and not shit (1%). Yeah.

So I don't know. Maybe this will turn out to not be as bad as it seems.

The main market for these tools are fanfiction and smut - it's a way to empower people who want to create their own personal fantasies or stories that nobody else would write, without having to be a good writer.
Personally, I like reading and usually prefer to read a book I like instead of a book I don't like, so the existence of more books I like would be cool. I expect I would still read human-written fiction, too.

Many authors are already "obsolete" in that they're not paid a living wage for the job of writing creative fiction. It's understandable to prefer being paid for it, and in an ideal world being obsolete would just mean you get to do it for fun and fulfillment and not worry about the money.

To add to your last paragraph: The median income for full time writers in the UK is below minimum wage. Most full time writers do it as a supplementary income only (the median household income for full time writers is above the average), and the vast majority of writers are not full time.

If you as a writer use a paid editor and cover designers, odds are it'll take you several years to break even today.

Unless you're "discovered" by a traditional publisher and they think they have the next Harry Potter on their hands, even being traditionally published means next to no sales for a large majority.

So it's already about fun and fulfillment for most authors. I went into writing (so far two) novels knowing the above, and did it anyway. That said, some do go into it thinking they'll make lots of money, or desperately looking for another income source.