[keiferski]

Not to be intentionally contrarian, but I have thought a lot about this:

I think good fiction is actually the antithesis of AI, fundamentally. LLMs are really only capable of combining different things into output. This can give you some creative results, but it is ultimately kind of limited. I cannot imagine that prompting an AI a thousand times would give you something truly original in the way many high-quality fictional stories are. The magic and absurdity of life is a necessary thing to create such works.

[jackbrookes]

> LLMs are really only capable of combining different things into output

> The magic and absurdity of life is a necessary thing to create such works.

These two points seem slightly conflicting. Why if a human is truly more creative than AI, must it experience "the magic and absurdity of life" if it can be creative without "combining things into output"? Is a creative human then just not splicing and recombining their experiences into their creative output? And therefore, if an AI had more data that mirrors that experience in some way would it be sufficiently creative to create something "truly original"?

My view is that intelligence and creativity is all just purely recombination. AI can do that well just as humans can. The gap is that the data humans have access to is much more analogue, emotional, and grounded in the real world compared to the data that AI is currently trained on.

[snakeboy]

> And therefore, if an AI had more data that mirrors that experience in some way would it be sufficiently creative to create something "truly original"?

An important distinction seems to lie in a human having original experiences to draw from, while an AI has only "data that mirrors" experiences.

[exe34]

> while an AI has only "data that mirrors" experiences.

True for a lot of current AI. With more robotic bodies, loads more access to cameras/audio/satellite/science measurements, an AI can have original experience on a level that can only be described as god-like compared to humans.

[kraai]

You could argue AI already partly has access to these measurements. You could easily tap into smart sensor data etc, same voor video/audio etc.

The part that AI can't (yet) truly replicate is 'feelings/emotions'. Experiences such feeling dread, loneliness, happiness. Or being anxious, nervous or having goosebumps. You can describe all these states but they are differently experienced for each individual and also dependent on each unique situation.

Ofcourse this also then comes down to how you exactly define emotions.

[exe34]

We don't know that Claude doesn't feel dread when it looks at my code. To be fair, these emotional circuits are the result of evolving in various environments. AI might well have completely new emotions that we just can't comprehend.

[keiferski]

And at this point you’re far off into science fiction, and beyond reasonable discussion of this.

Even then, it’s not really clear to me that an android with the exact appearance of a person will have the same experiences. They aren’t biological creatures with millions of years of evolution behind them.

[EGG_CREAM]

If it has a subjective viewpoint, i.e. sentience. Otherwise it still doesn’t actually experience any of those things, and can’t tell actually you what it’s like to experience those things. It can give you the data readings though.

[brazzy]

Wouldn't it be funny if they missing thing is actually the body, and having one to give the subjective viewpoint something to center on directly leads to emerging sentience?

Though I think there's at least one more important thing missing, namely the ability to form memories - to incorporate recent context into the model's weights. I don't think it's a coincidence that this is a very complex process in humans as well (involving sleep and dreams as neccessary mechanisms).

[rahimnathwani]

ChatGPT has had more conversations than you or I have. And it's conversed with people from a wider range of backgrounds in a wider range of situations.

[perrygeo]

Billions of conversations. Zero lived experience.

The difference is huge. An LLM conversation can contain within it a batshit crazy model of the world and still produce a plausible stream of tokens. No skin in the game. If you don't test your ideas against reality, you don't have a model of reality; you have a model of written human thought. It's critical to understand the difference.

[keiferski]

And? Can you tell me what it feels like to eat a hamburger by asking a thousand people about it?

[everdrive]

I think this is actually a tough one. Yes, I agree that an LLM can never experience this. But -- when a real human needs to translate that experience back into words, it can actually do so just fine.

In other words, when a real human needs to translate an experience into words to share it, A LOT of information is lost. The LLM might actually be better-suited for the the task in your example, even if the point you were driving at stands.

[keiferski]

I don't think it's all that difficult. Language is not a 1-1 match to reality. Ergo any entity which is inherently language-only, will fail to accurately describe the experience.

[lordnacho]

That's what we all did in school, didn't we?

Nobody did any original research, you didn't discover gravity, and you didn't participate in any historically significant events.

Yet we say that people who have been through education know about such things.

[keiferski]

That is descriptive knowledge. Not firsthand qualia.

You may know every detail of the American Civil War, from every book ever written on the topic. You may read every journal from every person that wrote one about it.

You do not have firsthand knowledge of what it was like to participate in the war. Any assumption of that knowledge is a (false) generalization of what you think a person “would have thought or felt” based on contemporary ideas of the personality, psychology, etc. Reading a journal entry about a soldier seeing his comrade killed is not equivalent to actually having that experience yourself.

[vitorfblima]

He probably could tell you something about it, based on other people's experiences. Doesn't mean he would actually know what he was talking about.

[keiferski]

He could tell you a collection of descriptions. Not actual knowledge firsthand.

Does reading an infinite number of cookbooks mean you know what the ingredients in each taste like?

[rahimnathwani]

Yes, of course.

[keiferski]

That’s a pretty lacking theory of knowledge. How can you know what taste is, if you’re never tasted anything?

If you read a thousand books about living in New York City, is that the same thing as the experiential knowledge that comes from living there in person? I certainly don’t think so.

[JKCalhoun]

"Is a creative human then just not splicing and recombining their experiences into their creative output?"

I do think so. The difference is that, even apart from synthesis, the creative human knows how to filter out the shit ideas from the clever, subtle ones—LLMs seem to just run with the first…um, thought that… um, comes out of their head.

I suspect knowing that you've on to something special probably does require "human experience", woe, melancholy, "the human condition", reflecting on the brevity of life…

[wanzg]

> My view is that intelligence and creativity is all just purely recombination. AI can do that well just as humans can. The gap is that the data humans have access to is much more analogue, emotional, and grounded in the real world compared to the data that AI is currently trained on.

Yes!

Aside from the more fundamental questions about the nature of cognition and what constitutes intelligence, there's a practical matter: humans have life to learn from. We have a lifetime of persistent memories, an identity, a social life, physical interactions with the world of many qualitatively different kinds, and we're taken on a lifelong journey into different studies, workplaces, and living environments by the biological need to survive.

Even if human creativity is in some sense also recombinative at its core, we have a very different set of inputs from LLMs. Even under the (almost certainly false) assumption that human condition is basically similar in structure to an LLM, we should expect some differences in outputs based on these profound differences in inputs.

[SkyBelow]

>These two points seem slightly conflicting. Why if a human is truly more creative than AI, must it experience "the magic and absurdity of life" if it can be creative without "combining things into output"?

I agree, yet I also think AI is extremely limited in terms of new creativity.

My guess is that humans operate on some deeper level. Much like the grokking model example where it easily trained on the training data, but only later did it really start to understand the deeper problem space and 'grok' the problem, I think LLMs are doing something similar with human thought. LLMs can throw more compute, but they are still lacking some introspection/drive that leads to the better creativity of humans. Grokking this from the training data might not be possible, or it might take some fundamentally different approach, or even an imperfection in humans that LLMs don't have (ego/drive?).

This isn't to say that LLMs could never do it, but the cost to do so might require magnitudes more scale in training times and size. Or it could be in the next model released. Or maybe they already can, but that gets trained out in favor of overall more general correctness accross all domains.

[whilenot-dev]

I think you're just neglecting the fact that LLMs only exist in the virtual world, whereas we're interacting with a world that's richer and has much more to offer. The virtual world is still just a mere subset of our world, but whether we want to perceive it as such is up to each individual. It certainly feels like you're conflating digital with analog, and a digital signal can still only approximate an analog one.

I still doubt that we have the knowledge to describe all of our human desires as mere results of our brain interacting with the nervous system either. There are many processes going on besides analysis and synthesis. If I plan on doing something extraordinary, I don't only sit there, think, and try to rearrange some pieces from memory. I'll go look around, touch, taste, re-explore and expand the perception my environment has to offer. Further, my perception can vary by the minute and I could even try to bring in a touch of chance whenever I feel like.

Wittgenstein's "whereof one cannot speak, thereof one must be silent" might be fitting here, yet language still continues to evolve.

[JKCalhoun]

"…LLMs only exist in the virtual world, whereas we're interacting with a world that's richer and has much more to offer."

I have a gamer friend that believes, having played 100's of 1st-person games, that he has had rich experiences because of it.

I reflect back on a day my family and I set out to hike through Arches National Park (still a Monument at the time, I believe) and the heat was such that my oldest daughter seemed in danger of going into runaway heat exhaustion. I began to panic as we were so far along on the hike already that even returning to where we parked seemed to pose danger for her.

There was little to no "coverage" (shade) along the trail and the heat and sun were intense that day—we had passed almost nothing. Finding the smallest of bushes to provide minimal shade for her, I scouted ahead on the trail to look for some kind of larger area where there would be enough shade, perhaps a breeze even, to allow her to bring her core body temperature back down.

Fortunately I found just that. (And it turned out that, in the news later, two hikers within a few hundred miles of us had in fact died that day as it was a record heat wave for the area.)

I think somehow that me and my daughters experiences that day, when actual death was perhaps on the table, was something a "virtual environment" can never really offer.

[keiferski]

Is a creative human then just not splicing and recombining their experiences into their creative output?

No, they aren't. I'm not sure why it's such a popular narrative that all art is just some kind of recombination of existing things; it's not. Actual lived experiences are unique and impact what is created. Novels written by people in post-Revolutionary France are not the same as novels written by post-WW1 France. Events in history shape culture and introduce new ideas and experiences that weren't there before.

The gap is that the data humans have access to is much more analogue, emotional, and grounded in the real world compared to the data that AI is currently trained on.

That's the problem with your argument, right here. Emotions are not merely portable "data," and framing this way is a problem from the start.

I don't think you get human experiences without actually being human. The most perfect LLM copy of a mind is just the copy of externalized output.

[card_zero]

It's often said that creativity involves reuse because that's clearly part of the process. It's like a warning against another narrative, where a reclusive genius outputs great works made from scratch, without being part of culture. Yet there's some truth to both. The thing is to be inspired by the culture and then be defiantly original and work away stubbornly in private and also talk to your peers and your audience. It reminds me of how we used to play as kids: oh you're doing that thing? I'm gonna do that thing too ... when you see the big reveal, you're gonna be very impressed! I'm doing it differently! Look at how I'm doing your thing that you came up with!

[JKCalhoun]

"I'm not sure why it's such a popular narrative that all art is just some kind of recombination of existing things; it's not."

I would like to see art that was "created in a vacuum" so to speak. I am not aware of any. A first time poet who had never read any poetry or prose or heard a tale told? A painter who had been blind since birth and just handed a canvas, oils?

(To be pedantic, the person you quoted said art was recombining their experiences which you took as existing things.)

[JKCalhoun]

"…truly original in the way many high-quality fictional stories are."

The older I get, the more "nothing new under the sun" returns again and again.

[dimitrios1]

"I will say nothing new!"

- St John of Damascus

[amarant]

Agreed! I find AI to be pretty decent at documenting and explaining technical stuff, which is a subgenre of non-fiction.

I once tried telling it to write a fantasy novel for me because I didn't know what to read next. Gave it some examples of books I have enjoyed in the past so it would have an idea of my taste...God was that awful. Absolutely terrible idea.

AI is not coming for fantasy authors jobs any time soon!

[brrrrrm]

Models are increasingly showing their ability to extrapolate into the human unexplored (math proofs being the most apparent). What gives you confidence the absurdity of life is uniquely difficult for models to source?

[eddythompson80]

Mathematical proofs are hard to come to, but easy (or “cheap”) to verify. Also having a math proof processed through a transformer to derive an output that matches existing proof patterns kid of makes sense. The former allows you to generate hundreds or thousands of iteration and cheaply verifying its correctness until you hit the right randomness level. Generating thousands of fiction books then verify them doesn’t really work.

Most good fiction authors are trying to communicate a story that’s forming in their minds. Even when they get stuck, they are usually not stuck because they just don’t know anyone to ask. It would’t be their voice anymore. Some do seek input and those might experiment with llms for ideas. Using llms for research, technical questions, grammar, synonym sentences, word selection, etc makes sense too.

I’d agree with OP for people generating full stories through llms. While I say that, I know a very young person who told a while back (like early ChatGPT 4 time) they use it to write fan fiction for books or characters they like.

[Jtarii]

Being able to write a series of logically correct sentences down is not the same thing as intuiting human emotional experience. Or even related.

[keiferski]

Math seems entirely unrelated to the human experience to me, at least in the sense of “this thing happened to me and I will transmute it into a work of art.” Math is pure abstraction.

[sigbottle]

I will readily admit that I didn't read Dostoevsky, and I don't continue to read Dostoevsky, because I've compared him against a swath of other fiction writers and authoritatively determined with some objective reproducible standard why his writing has been impactful and changed my life. I heard about Dostoevsky from a lot of people talking about depression when I was depressed, he resonated with me. Then when I came back to him half a decade later, having resolved more of my issues, I was able to evoke similar feelings but through a more mature lens.

Do I reflect on the objectively correct structure of the story? Do I "agree" with everything Dostoevsky is saying? Not really.

I don't know if this offends literature people or not. But this sure as hell is incompatible with the AI bro paradigm in which all thought can be objectively reduced to some shared standard to benchmark on.

My position will be read into as deep irrationality from many opposing positions, but it's fine.

Now, none of this doesn't "preclude* the ability for AI to have the same kinds of network effects. Of course it can - that's what I've been saying for the longest time. But don't confuse, becoming integrated with society, with this ideal of superhuman X.

Now of course, you could argue, "there is no such thing as anything else besides becoming integrated with society. There is no other standard that works." Very good! Darwinian-adjacnet positions are interesting. This is a position - but then don't claim it's equivalent, or god forbid, the exact same, as the "super AGI" ontology. These require fundamentally different assumptions about what justified what, and an attempt to try and pretend otherwise is sophistry.

[oh_no]

I have also thought about this a lot but have less developed views on fiction (short version: I don't currently want to read an LLM-written novel, but I expect that will change in the future and there will be some I enjoy)

But I want to say that non-fiction books are a lot more resistant to LLMs than you might think. I'm reading two non-fiction books currently. And both (https://www.goodreads.com/en/book/show/17801.Underground, https://www.goodreads.com/en/book/show/44824581-a-k-pop-live) could not have been written by an LLM. One is based on in-person interviews that were hard to set up, and the is based on a log of sources a number of which are going to live events.

Not every book fits this pattern of course. But even a more straight forward book (https://www.goodreads.com/series/139929-designers-dragons) will often rely on sources that are not easily accessed by an LLM, they may not be digitized, or they may not be accessible in your common digital collections.

I 100% think LLMs can do good synthesis work, don't really think we have the tooling/checking in place for them to write a competent non-fiction book yet, but that will certainly happen, but I still think the best and most interesting non-fiction remains firmly in the realm of humans.

(Also, a lot of non-fiction is memoirs, travelogues, etc. which LLM can't really replace unless you're dictating to it.)

[deadbabe]

While you are right that humans can probably find some original ideas that are not just combinations of inputs, the universe of things that humans will find interesting to read is not that vast, and that’s why there are only really like 7 types of stories, and those are the ones that survive through a process of natural selection.

Everything else is just not that interesting to a human.

[slim]

  LLMs are really only capable of combining different things into output

what makes believe humans can do more than this ?

it's a real question. I've seen a lot of people stating that the human mind "is more than this" as if it was obvious

[Jtarii]

I would say the inherently chaotic and continuous nature of the human brain can create novelty in a fundamentally different way than LLMs can.

Like a human can reach into the chaos of their own emotions and pull something unique out. There is some layer of the human experience where ideas can ferment and change based on all your memories is weird ways, and LLMs completely lack this, which makes things LLMs create very blunt and obvious. Like there is no nuance to anything they do.

[gue-ni]

Well since human culture has developed from very limited stone age culture into what we have today, we must conclude that at least _some_ people are capable of generating something that is genuinely new.

[lordnacho]

So now we'll have to investigate what "original" means?

From what I can see, AI is absolutely able to create new work. Things that are not in the dataset, but look like they could be. This is pretty much what generalising means, they can make a wide variety of things that aren't actually in the original set but can hide in that set pretty well.

When you are an original piece of culture, what do you mean by it?

[keiferski]

Can an AI write a story about living as an artist in NYC in 2026, the cultural environment, etc. without that information being added to the dataset immediately as it happens?

It seems pretty obvious to me that there are cultural trends which are happening instantaneously and aren’t merely some sort of consequence of past historical data.

[seanmcdirmid]

Ideally the AI would get none of its world knowledge from its model weights and would rely entirely on context. This is just an ideal though, we haven’t figured out how to completely separate world knowledge from the model.

[lordnacho]

But how could something be a cultural trend without some sort of anchoring in history?

I don't see why a computer (or a person) couldn't make up a plausible sounding story about a NYC artist of the present day? You would just reference things that don't change that much (artists having a boheme type existence, wild inequality in who can live off their art, NYC being a sort of place where people go to become successful).

[awongh]

> LLMs are really only capable of combining different things into output

In terms of how writers think about creativity this doesn’t reflect reality- there’s the old writer’s saying that there are only 7 stories (and other variations of this idea) and that writing is about creatively remixing these well-worn ideas.

The LLM’s capability to write acceptable fiction and nonfiction is coming soon if it’s not already here. (I think right now it still needs some high level input from humans, depending on length and topic)

It’s clear to me that, especially as LLMs get better and better, there’s only one real difference- it’s that I don’t want to hear what an LLM has to say. I’m only interested in what other humans have to say. I feel the same way about AI music- even if it sounds ok, even as good as the kind of unoriginal pop music that’s not AI made (but is a kind of it’s own slop) or a bad committee made hollywood movie, it’s still fundamentally more interesting than something that’s been generated. Even if it’s 100% fiction it’s still based on a real human’s life experiences.

[keiferski]

This is a bit like saying “there are only five fundamental flavors” and everything is some combination of them.

Maybe, but that has basically zero effect on the experience of actually eating food. Just because the form may be the same doesn’t mean an LLM will be able to supply original content.

[awongh]

I’m not sure I understand the analogy. You could say that the reason why artificial flavors work in food is because we know how to recreate flavors from first principle chemistry. The dislike of the artificialness of cheetos also doesn’t stop many many people from eating them.

I would say human made slop writing isn’t any better or worse quality than LLM writing.

Knowing if it was made by a real person makes a difference to me, but I also know it won’t to everyone.

[keiferski]

Just because the form of a story is the same as another story does not imply that the content is interchangeable.

I could write you a story right now about being an expat living in Poland in 2026. It’s unclear to me how an LLM would have the cultural knowledge or experience to write such a story that is actually accurate and up to date, without already having something like my story already in its dataset.

[lordnacho]

Yeah but you would just be doing the same thing: mashing up some known information about humans and Poland. You would pick plausible names like Marek or Piotr, plausible places like Warsaw or Gdansk, and some random flavour of story that is either completely general across cultures (hero's journey) or some local tale from Poland.

Shakespeare did the same, so I wouldn't blame you for it.

[keiferski]

I am an expat in Poland, that’s why I used it as an example. The point being that I have knowledge of that experience that is not written down in an LLM dataset.

[ModernMech]

Which would you rather watch:

Superhero Wars Trek 42 - the latest franchise installment, billion dollar Hollywood budget, financed and produced by billionaires, audience tested committee-made script, lots of people involved, designed primarily to push merchandise.

Personal Story - a movie about someone's personal reality or fantasy, something typically not made by studios because the story isn't profitable to a wide audience, painstakingly generated and edited by one person using AI, designed primarily to tell a story.

I put up a different strawman because your strawman is too beatable. As humans, of course we're all interested in what other humans have to say. But you haven't explained why you feel people can't use AI to say things, or why AI output can't be based on a real human's life experiences.

Personally, it seems like movie studios are the ones with only 7 stories to tell, and I've seen the movie and its sequel, the prequel, the requel, the remake, the gritty reboot, the musical, and the animated series. I'm excited that AI will let people skip the studio gatekeepers to tell some new stories for a change.

[newguy1000]

you have a bad imagination then

[d1l]

Exactly. Shows us what is true in the sense of the experience of being a human. It is resilient against slop because good literature has experiential depth. Maybe LLMs learn to emulate this some day, but I doubt it. Not enough money in getting it right.

[sourcecodeplz]

that is how humans do it too! they combine things, ideas.

you can make LLMs do that to