[Apocryphon]

From the comments:

> I will concede that using LLMs in software development is a different category from fiction, insofar as the languages and APIs are much smaller and more tightly constrained and in software you actively don't want long-range dependencies between elements (something you often do want in fiction, i.e. put a gun on the mantlepiece in part 1 of the book, pull the trigger in part 3).

[bonoboTP]

It can definitely plan such things. Because agentic systems don't have to output a book all at once. The mainstream still lives 1-2 years in the past.

Agents can plan out a narrative draft, write it out section by section, maybe out of order, then tweak it, plant clues if it wants to, then trigger it in another chapter file etc. Then read the whole thing, reconsider, etc. It can iterate. I'm not saying it will be great literature, but the problem isn't the inability to create long range references.

My bigger point would be this: arguing from such mistaken technical ideas, when not having any idea about the tools is a sign that people's real problem is none of those technical limitations, but Western culture has lost the vocabulary to express their true sentiment. They are grasping for words but can't reach for "soul" and other religious terms because that would sound ridiculous in a secular modern world. But they clearly want to express that and we'd be ahead if we could discuss in such actual terms. Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not. Because there is something mysterious and important about the human spirit. (the same vocabulary-inaccessibility is plaguing other culture war issues too). People will rather talk about fake water consumption problems than touch the topic of the human soul or spirit.

[ben_w]

> but Western culture has lost the vocabulary to express their true sentiment. They are grasping for words but can't reach for "soul" and other religious terms because that would sound ridiculous in a secular modern world

Ironic to implicitly accuse Stross of this; as I recall, he's one of the voices describing The Singularity as Christian Eschatology/Book of Revelation/the rapture for atheist nerds.

> Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not.

One of his own paragraphs explains his non-financial perspective and reason:

  I do not want or need a large language model to write my fiction for me. I write fiction compulsively—before I was published I wrote for many years as a hobbyist—so why on earth would I pay someone else to take my fun away?

I may not have quite the level of obsessive desire to write as he has, but I have some, and it is different from the desire to simply consume someone (or some*thing*) else's work.

Without criticism, he is also clearly upset about the economic impact this will have on him (he says specifically he is "one of the parties to the settlement in the class action lawsuit against Anthropic AI for pirating ebooks to train their LLMs" and "train an LLM that is intended to compete for revenue with the authors of the works they stole"). Writing is not a high-earning job (with the exception of a few superstars who may as well be lottery winners), and he has written in support of very long copyright durations that can be inherited.

> People will rather talk about fake water consumption problems than touch the topic of the human soul or spirit.

For a living, he writes about Cthulhu and their ilk doing mind control; I want to say I don't think this would put him off, but if it did he'd hardly be the only professional author who can write a fictional story about something without being able to apply such ideas to rhetoric. (He'll hate being compared to Rowling here, but consider how Rowling hates trans people while her fiction has a variety of shape-shifting spells).

[bonoboTP]

You need better arguments than "you don't like to use it yourself and prefer writing yourself" or some metaphysical voodoo-like theory (analogous to tribes that refuse to be captured on a photo) whereby feeding your words to a training algorithm somehow corrupts your pure Platonic words or whatnot. Your utterances are not yours to grab on to. Copyright is about expression and derivatives. You could never copyright your style, or else any music band that copies the general style of another band could have been sued and then the development of music would also have gone quite differently. No, ideas, styles, claims, knowledge are by default part of the commons. The narrow carveouts are for incentivizing more creative work and are targeted and specific, time limited. You can't generally, by default, restrict what others can do with ideas that you've put out. Patents are a narrow exception. And copyright for non-transformative derivative works or copies. And trademarks to protect consumers from mis-identifying the creator of something.

-----

> clearly upset about the economic impact this will have on him

So readers will read AI generated books because they are better than his writing? People can't invest ten bucks or however much into reading a good book? How about piracy? It exists. I bet I could download all his books in less than 5 minutes. If someone insists on not paying for that book, they can easily get it.

> he has written in support of very long copyright durations that can be inherited

This whole copyright and intellectual "property" idea is extremely new in human history. More people should know this fact. This has not been how people thought about culture before the ~1800s, or, well before the printing press. Copyright is a narrow exception to the general rule that people are free to listen to others and use the ideas and spread the ideas. There are exceptions like trade secrets and various dangerous information about nuclear topics and whatnot, but generally preventing others from talking about what they heard is tyrannical. If anything, copyright should be reduced to something like the creation of the work + 20 years. Not lifetime +X. That's plenty enough to be the sole distributor and to block others from building on top of it. After that it's time to let people use culture as it's always been. Changing, mixing, remixing, retelling, combining, etc.

The whole thing is often wrongly framed, with verbs like "consume", like "consuming content", which is just Orwellian language to describe having your eyes or ears open and remembering what you saw and heard. It doesn't consume a pdf to look at the words and read them. It doesn't consume a film to look at it being projected on a canvas. It doesn't consume a painting to step close and look.

[crancher]

I tackle these ideas/questions head-on using Claude to help book-shape the result. https://spine.zice.app/jeremy/the-smaller-infinity In my view, the accumulating density of useful language/token patterns is the real story, not who/what is processing them. There is no "why" other than the universe's physics permit such accumulation, but that doesn't make the "how" any less interesting. You can click the header and copy the whole thing to clipboard, ask your favored agent for summary/conversation.

[drdaeman]

> Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not.

I sure would. If you want a religious tone to it… It would be a signal suggesting that the human spirit(tm) had finally transcended.

If humans can someday build a machine that can truly carry a (trans)human mind, and let some future souls live unconstrained by all the quirks and limitations of our preexisting biology, exercising more control of their own lives - that’d be wonderful.

I obviously have no idea wherever language models can lead to such dream, of would be a dead end.

[dwb]

I think attitudes like this are traitorous to the human species. You’re not talking about a transcendence, or at least, not one that I recognise. Our biology is an innate part of us - to improve, to fix, sure, but not to cast off. Who is directing this “transcendence”, and who benefits? I absolutely do not want a religious tone to any of this. Awful vibes.

[crancher]

I find this "traitorous to the human species" interesting. You could not form these thoughts without an accumulated knowledge stored/accessed outside human biology. No other species forms such explicit thoughts because they lack access to that accumulated pattern of useful sequences. We've already "transcended" our biology in that sense. Who is directing... that becomes interesting in new ways once outside-a-body sequences start to accumulate. My guess is that it's that accumulation that sets the trajectory, just as DNA's "accumulation" of useful sequences produced forms fit for the local environment(s).

[dwb]

Sure, we're cyborgs, but still very much of the flesh, and I think we have to draw a line somewhere.

[ben_w]

> Our biology is an innate part of us - to improve, to fix, sure, but not to cast off.

Right now, necessarily so.

The more mysteries that science solves, the more we unweave the rainbow*, the worse the tension between rationality and our sense of what we are. Am I the atoms, or the pattern betwixt? Am I the present moment alone, my memories mere shadows of past patterns and my current form doomed to be a shadow to a future pattern of these self-same atoms?

Sometimes even the question is unbearable.

* https://en.wikipedia.org/wiki/Unweaving_the_Rainbow

[dwb]

There are many mysteries of the universe, but who we are, to the extent that I am rejecting the comment I replied to, is not one of them. I would love to see how humans explore reality and being (if we survive!) as time goes on and we figure more out, but I hope we do so as a whole species and not through the designs of a handful of zealots.

[dgellow]

What are you talking about… You can use the word soul without believing in a religious concept of soul. It’s done all the time in secular societies. It’s even a very common term used to describe the problem with LLM generated content, that it lacks soul or human spirit, it lacks the human meaning, intent

[bonoboTP]

It's rare. It's mostly, "but it will never be able to do X" (when it can already do X), or it uses too much water (while having no idea or reference point to what amount is much), or other climate things, or non-progressive bias, or copyright (when training is fair use) etc.

Just be upfront. 1) It threatens your livelihood that people can get the thing done faster and cheaper than what they used to pay you, 2) our relationship to an AI shaped entity is spiritually risky and a esthetically off putting, and we have to talk about golems and the tower of babel and stuff. If you think I'm being incoherent, you're in for a ride. Society will have to come to terms with major major issues, not just job replacement and bad slop art.

[user43928]

I think it is true that people who obviously dislike AI for ideological reasons often lead with ecological concerns or doubt about the capabilities, leading to an unproductive discussion.

[dgellow]

Have you considered engaging on those ecological concerns? There is quite a lot to dig into and can be pretty interesting. For example, try to project how much electricity will be needed to power all the datacenters in construction, then compare that to the existing grid and infra, then figure out the cooling capacity. If done well you will see that we have a pretty massive resource problem here

[user43928]

I had a look now, and apparently we are projected to increase by 2.3x datacenter power consumption until 2030 from 415 TWh to 945 TWh.

That is going from 1.5% of global electricity use to 3%.

It is less than I expected.

Maybe if AI does work out better than expected for knowledge work or becomes applicable to robotics and fields like construction, we are going to see a larger buildout.

[bethekidyouwant]

So electric cars but bad