[vertis]

Prompt: Pretend you are William Gibson. Pitch your next book to your publisher.

GPT-4: > Dear [Publisher],

> I hope this letter finds you well. As you know, my work has always strived to explore the intersection of technology, society, and human nature. With that in mind, I am excited to present to you my latest novel, "Neural Drift."

> Set in the not-too-distant future, "Neural Drift" delves into a world where the lines between our reality and the digital realm have all but vanished. The story revolves around our protagonist, Elara, a skilled hacker and data courier who navigates the vast neural network known as the "Mindscape" – an immersive digital landscape that has become the primary means of communication, commerce, and entertainment for the global population.

> In this future, the Mindscape has become an integral part of daily life, and the distinction between the physical and digital worlds is all but gone. Society is now heavily reliant on neural implants, which allow users to access the Mindscape and interact with the digital world through

Anthopic Claude-v1 (which wouldn't answer at all if I didn't put pretend in front):

> Here is my William Gibson-inspired book pitch:

> The future is here—it's just not evenly distributed yet. In my new novel, Pattern Recognition, I explore the bleeding edge of culture, technology, and society through the eyes of Cayce Pollard, a "coolhunter" who has an uncanny ability to spot trends in the wild.

> Cayce is hired to track down the creator of a series of mysterious film clips being uploaded to the internet. Her search takes her from London to Tokyo to Moscow, where she encounters a dangerous mix of hackers, hipsters, and Russian mobsters. Along the way, Cayce begins to realize the film clips seem to have a strange predictive power—they appear to show events that have not happened yet.

> Pattern Recognition is a stylish, fast-paced techno-thriller that examines how technology shapes culture, and how culture shapes us. It blends Gibson's trademark cyberpunk sensibility with a

YES, it did just regurgitate an existing Willam Gibson book

[alehlopeh]

But you didn’t ask it to pitch a new book. At one point, Pattern Recognition was actually his next book, and there’s probably a lot of text out there discussing it as such.

[vertis]

Sure absolutely, but would a human make this mistake? You can't pitch something that already exists (I mean in this context we're pretending, so maybe we can pretend we're pitching that book). But a human (and GPT-4 does) assumes you want an original book. It's still an assumption.

I was playing around with GPT-4 making it take the questions for an Economics exam the other day and it was missing a lot of context that a human would get or assume. It was still much better at economics than me, but was going down "wrong" paths that a human wouldn't go down.

If you told it to explain the the question and list assumptions you could work out where it was going off track, but it lacked the ability to step back and recognise it had taken a nonsense path.

[andybak]

> but it lacked the ability to step back and recognise it had taken a nonsense path.

Errr. Yes! We're at the stage where you should be amazed how capable these things are considering their fairly brute force nature. Being surprised at the limitations is a peculiarly - dare I say? - human response?

Usually the workflow involves:

1. Write a naive prompt

2. Gain some insight into AI's behaviour

3. Improve prompt and repeat 1.

Of course they lack insight. Of course there's no "meta-reasoning". What is astonishing is how often a glorified Markov Chain gives the impression that there is both.

[vertis]

It's all astonishing. I don't think it's so much surprised at the limitation as trying to explore the edges. Enjoying finding some of the emergent behaviours.

[creativenolo]

It seems like your issue is that this is not human, and it does not do what a human would do. This is a high expectation and, possibly dangerous, assumption.

Shows how this tech is such a double edged sword for user value, expectations and assumptions.

[vertis]

No, I'm well versed in the limitation of the tech. The comparison to what a human can do is just an interesting comparison. It's a feat the GPT-4 model can manage in this context. GPT-4 is also exhibiting emergent behaviours that are not found on models with smaller numbers of parameters, it is these emergent behaviours that make it seem more 'human' in it's ability to produce content, even if the "reasoning" about it is all an illusion.

Even here there are limitations, which was my intent with mentioning the Econ exam.

Aside: It's also possible to get different answers by making the LLM simulate being different people (a student, or a professor).

It is very easy to slip into thinking about ChatGPT in terms of actually thinking, but then again our understanding of the human mind has led to theories that conscious decision may well be an illusion as well (if brain scans showing that the decision is made before the conscious mind is aware are anything to go by).

At the moment though, these are just tools, with varying degrees of usefulness.

[carrolldunham]

It's not a mistake. It's just ambiguous - you and the OP (rather than "humans") share some contextual assumptions that to you make it imply one meaning. If someone asked me with no context to 'be william gibson pitching your next book' I might answer just the same as gpt did

[vertis]

It's a fair point, and some of the context that's missing for the LLM is just present in the background for humans. You wouldn't need to be told, it would be whether you were in an acting class or creative writing class (these are weak examples).

We get so many context clues that aren't present in current LLM text interactions.