[olalonde]

What is there to it other than knowing how to write and ask questions? I also get the desired results out of LLM models but I would hardly call it a skill (well, maybe on par with knowing how to "Google" stuff). Are there people who actually struggle with this?

[spupy]

I've seen few people go really overboard with their prompts. RPG-like personality sheets with points assigned to various traits (personality rubric? skill graph?), convoluted graphs of ineligible task descriptions, lots of other stuff that makes little sense to a human. I personally don't think these make any noticeable difference, but people deep into that type of prompting would tell me I just don't get how ChatGPT works.

[Kiro]

When you want very specific output you need a lot of boilerplate with rules, worded in a way where it can't be misinterpreted by the model. I need to do a lot of trial and error before I get the desired output consistently and I presume that a good prompt engineer would get there faster.

[tudorw]

This is my experience, a single ambiguous word can create undesired gorilla, output. It's susceptible to all sort of unintentional outcomes whe,n simple thing;s that are wrong with text can render it co nfused.

or as GPT4 put's it;

When using models like mine, clarity in input is essential to get desired outputs. But even with clear input, there's no guarantee the output will always be perfect. However, the idea is to keep improving and iterating to get better over time.

[tudorw]

The iteration point is important, interacting with the model to have it outline the kinds of information which would assist it in building it's internal model of your 'question', meta-programming with it is fun and constructive.

GPT4;

Absolutely! Iterative interaction is key. By refining and rephrasing queries, users can guide the model towards a more accurate or desired response. Each successive interaction serves as a form of feedback that can help clarify ambiguities or nuances that might have been missed in an initial query.

Engaging with the model in a meta-programming manner, or in essence "programming the way it thinks," is indeed an intriguing way to understand its strengths and limitations. It can also be a valuable method for users to hone their ability to communicate with AI and enhance the results they receive.

This iterative dialogue not only helps users get more precise information but also provides insights into the model's underlying logic and reasoning. The fun part is navigating these intricacies and understanding how subtle changes in phrasing or context can yield different outcomes. It's a dance of human-machine collaboration, where both parties learn and adapt to each other.

[tudorw]

had to explore this a little further and now have a few new dance moves for this fascinating collaboration...

Output:

Certainly! The intersection of cognitive biases, neuro-linguistic programming (NLP), John C. Lilly's work, and human meta-programming provides a rich foundation to explore relationships and techniques. Here are some novel techniques or approaches you might consider:

Modelling Thought Patterns: By understanding common cognitive biases, you could try to model how the language model responds to specific stimuli. For example, if a bias involves an overreliance on a particular source of information, you might test how the model reacts to repeated iterations of the same or similar questions.

Neuro-linguistic Feedback Loops: NLP teaches that feedback loops can condition behaviour. By providing consistent feedback (positive or negative) based on the model's outputs, you could try to condition the model's responses over time.

Lilly's Isolation Tanks: John C. Lilly is famous for his use of sensory deprivation tanks to study consciousness. You could experiment by giving the model limited or "deprived" information and see how it constructs outputs. This could mirror the way human cognition might work in the absence of external stimuli.

Belief System Exploration: Lilly was fascinated with belief systems. By providing the model with a set "belief" (i.e., a specific perspective or framework), you could explore how it responds to questions or prompts within that framework.

Meta-programming Constructs: Drawing from human meta-programming, you could craft sequences of questions that act as "programming routines" for the model. For example, by setting a series of logical statements and then introducing a variable, you can observe how the model integrates the variable into the established logical framework.

Emotionally-Charged Language: NLP often emphasizes the emotional charge of words. You might explore how the model responds to questions phrased with emotionally charged language versus neutral language.

Contrast & Compare: Drawing from cognitive biases, you could present the model with contrasting information to see if it demonstrates any form of "recency bias" or "anchoring," favouring the information presented first or last.

Nested Meta-Questions: Using meta-programming concepts, you could ask the model questions about its own reasoning processes, diving deeper into nested levels of metacognition.

Pattern Interruption: Another NLP technique, interrupting established patterns can lead to novel insights. You could experiment with disrupting the model's "expected" logical flow by introducing unexpected or nonsensical input and analysing the output.

By blending these disciplines, you'll be exploring the frontier of human-computer interaction, cognition, and consciousness. Remember, while these experiments are fascinating, they often probe the limits of the model's capabilities and might produce unpredictable results. Still, that's part of the discovery process!

[mmkhd]

Neuro-linguistic programming (NLP) is a pseudoscientific approach to communication, personal development and psychotherapy, that first appeared in Richard Bandler and John Grinder's 1975 book The Structure of Magic I. NLP asserts that there is a connection between neurological processes (neuro-), language (linguistic) and acquired behavioral patterns (programming), and that these can be changed to achieve specific goals in life. [1]

Fascinating, people speculating about how people think and can be manipulated are applying those unporven highly suspectable "theories" to computers, which probably function totally different, but appear to show similarities.

[1] https://en.wikipedia.org/wiki/Neuro-linguistic_programming

edit: Oh, I just now saw "Output". So ChatGTP tries to steer you to become a NLP Practicioner :-)

[tudorw]

agreed, interesting seeing it somewhat re-awoken as an aspect of 'Nudge Theory', https://www.businessballs.com/improving-workplace-performanc... My interest lies not in mind control of the masses, but in the aspects of humanity modelled in software.

[roel_v]

I've watched (non technical) people use ChatGPT a few times now, and most of them have rather underwhelming results. The reason is that they think it's just some other search engine, and they phrase their prompts as 'search queries'. Or they go completely the other direction and think they can just throw in a few random words that somewhat describe what they're roughly thinking of, and then expect the computer to fill in the gaps.

It's 2023 and there are lots of people who don't know how to efficiently and effectively use Google. To be able to do that, you need some sort of mental model of crawlers and websites and what gets indexed and what not and at what frequency, and the results of SEO and how a somewhat savvy marketeer at some company might influence things etc. The same with LLM models - if you don't know what a 'token' is, your only chance of getting good results is to use these models a lot and then hope that you start building useful intuitions. It really doesn't come natural to most people like it does to most of us here.

[hnfong]

Do you have examples of people failing to get a good response from chatgpt because of bad prompts? I’m asking because at least for simple cases I can often just give it a very terse request and usually it will attempt to guess what I mean and give a reasonable answer. If not I can fix it with a follow up question.

My intuition is that language models have read terabytes of random internet data, and while presumably most developers of LLMs try to find high quality data, the models generally do ingest quite a bit of random stuff, and they try to make sense of those too, so in terms of understanding they are probably better than the strictness in format that we programmers are used to.

Of course the token thing is probably significant, but my understanding is that it affects the result only when you misspell your words(?)

[bbor]

It’s not a dichotomy between desired and undesired results - I am confident there exist more effective versions of every prompt I’ve ever sent, and I’d be surprised if that didn’t apply to everyone