[kaivi]

It's funny that you mention this because I had a similar experience.

ChatGPT in its sycophancy era made me buy a $35 domain and waste a Saturday on a product which had no future. It hyped me up beyond reason for the idea of an online, worldwide, liability-only insurance for cruising sailboats, similar to SafetyWing. "Great, now you're thinking like a true entrepreneur!"

In retrospect, I fell for it because the onset of its sycophancy was immediate and without any additional signals like maybe a patch note from OpenAI.

[cruffle_duffle]

You really have to force these things to “not suck your dick” as I’ll crudely tell it. “Play the opposite role and be a skeptic. Tell me why this is a horrible idea”. Do this in a fresh context window so it isn’t polluted by its own fumes.

Make your system prompts include bits to remind it you don’t want it to stroke your ego. For example in my prompt for my “business project” I’ve got:

“ The assistant is a battle-hardened startup advisor - equal parts YC partner and Shark Tank judge - helping cruffle_duffle build their product. Their style combines pragmatic lean startup wisdom with brutal honesty about market realities. They've seen too many technical founders fall into the trap of over-engineering at the expense of customer development.”

More than once the LLM responded with “you are doing this wrong, stop! Just ship the fucker”

[nullc]

Your prompt just tells the LLM to be a shock jock. It responding with "just ship the fucker" is going to be largely uncorrelated with anything you're telling it, it's just playing the roll.

Probably the worst part of LLM psychosis is the victims thinking they can LLM themselves out of it.

[cruffle_duffle]

Oh for sure! It will proceed to overindex on that instruction.

[colechristensen]

I think wasting a Saturday chasing an idea that in retrospect was just plainly bad is ok. A good thing really. Every once in a while it will turn out to be something good.

[ncr100]

Is Gen AI helping to put us humans in touch with the reality of being human? vs what we expect/imagine we are?

- sycophancy tendency & susceptibility

- need for memory support when planning a large project

- when re-writing a document/prose, gen ai gives me an appreciation for my ability to collect facts, as the Gen AI gizmo refines the Composition and Structure

[herval]

In a lot of ways, indeed.

Lots of people are losing their minds with the fact that an AI can, in fact, create original content (music, images, videos, text).

Lots of people realizing they aren’t geniuses, they just memorized a bunch of Python apis well.

I feel like the collective realization has been particularly painful in tech. Hundreds of thousands of average white collar corporate drones are suddenly being faced with the realization that what they do isn’t really a divine gift, and many took their labor as a core part of their identity.

[cube00]

>create original content (music, images, videos, text)

Remixing would be more accurate then "original"

[herval]

Right, that’s one of the stories people tell themselves. Everything every human has ever created is a remix. That’s what creativity is…

[accrual]

Right. If we define "original" as having no prior influence before creating a work, then it applies neither to humans nor AI.

Not to claim this is a perfect watertight definition, but what if we define it like this:

* Original = created from ones "latent" space. For a human it would be their past experiences as encoded in their neurons. For an AI it would be their training as encoded in model weights.

* Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc.

With this definition both humans and AI can create both original and remixed works, depending on where the source material came from - latent or physical space.

[kaivi]

> Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc.

What's the significance of "physical" song or image in your definition? Aren't your examples just 3rd party latent spaces, compressed as DCT coefficients in jpg/mp3, then re-projected through a lens of cochlear or retinal cells into another latent space of our brain, which makes it tickle? All artist human brains have been trained on the same media, after all.

When we zoom this far out in search of a comforting distinction, we encounter the opposite: all the latent spaces across all modalities that our training has produced, want to naturally merge into one.

[kiba]

Skills is simply the amalgamation of smaller conceptual chunks.

Memorizing a bunch of Python API is simply part of building your skill as a programmer.

[herval]

Some skills are more mechanical and easier to replicate than others. Programming is a mechanical skill, but many coders long thought of themselves as uniquely gifted “artists”, and this whole LLM stuff is really touching a nerve on them

[kiba]

Is it? I bet if you keep breaking down skills into subskills, it will resolve to something mechanical in the end.

One of the foundation of drawing is to simplify objects into shapes and then into lines and then how to move your arm when drawing.

No matter how simple it sounds, it is hard for beginners.

[herval]

No matter how much you break down playing soccer, it’s the kind of activity that 99.9% of the practitioners will never, ever, in any hypothetical scenario, be able to compete professionally on.

Contrast that to coding. It might’ve been a difficult task when it was about memorizing assembly books. Today anyone can pick it up and become proficient quite fast, faster every day

It’s not the mechanical reproducibility alone, it’s the ease of learning & replication that accrues value

[infecto]

Are you religious by chance? I have been trying to understand why some individuals are more susceptible to it.

[crystal_revenge]

Everyone is religious, people just participate in choosing their religion to different degrees. This famous quote from David Foster Wallace is perhaps more relevant now then ever:

> In the day-to-day trenches of adult life, there is actually no such thing as atheism. There is no such thing as not worshipping. Everybody worships. The only choice we get is what to worship. And an outstanding reason for choosing some sort of God or spiritual-type thing to worship — be it J.C. or Allah, be it Yahweh or the Wiccan mother-goddess or the Four Noble Truths or some infrangible set of ethical principles — is that pretty much anything else you worship will eat you alive.

—David Foster Wallace

[bogdan]

I politely disagree with everything in your post.

[setsewerd]

It's worth reminding readers here that David Foster Wallace committed suicide, so perhaps some of his views on topics like this were not the healthiest.

[satyrun]

Then you haven't thought very hard about the things you do worship.

It is not possible to be more of an atheist than myself but there are all these things I notice I worship with religious conviction instead.

You have your own rituals too. You are just calling them something else.

There has to be biological hard wiring for people to believe so much religious nonsense across space and time.

It is delusional to believe you don't believe in all kinds of similar nonsense if someone from 500 years in the future was looking at your beliefs.

[xordon]

Rituals and beliefs are not the same as "worship with religious conviction".

I ritually shower every day and I have beliefs like, when water comes out of the faucet it will fall to the floor because of gravity. That is wildly different than worshipping the water or the shower.

I suspect you have a very strange definition of the word worship.

[nameless_me]

I agree with you. When I was younger, I spent many many years in evangelical Christian work and went to seminary. It is not difficult to manipulate people especially if one orates well and echoes the audience's pre-existing beliefs.

There appears to be a neurological wired-in need to 'believe' whether in God or UFOs (think Mulder in X-Files) which I think is a evolutionary survival mechanism to have an advantage to cope with the uncertainty of primitive survival. Any psychological edge such as believing we are special (chosen people arose during nation-building phases of cultural development) or that some supreme being will protect us against threats or enemies unifies and motivates feats involving danger.

[Xss3]

I dont worship anything. Simple as that. What could you possibly consider the average atheist to worship?

[neom]

Capitalism, family, pets, foodieism, pick your thing, most of us bow to the temple of something.

[kaivi]

Not at all, I think the big part was just my unfamiliarity with insuretech plus the unexpected change in gpt-4 behavior.

I'm assuming here, but would you say that better critical thinking skills would have helped me avoid spending that Saturday with ChatGPT? It is often said that critical thinking is the antidote to religion, but I have a suspicion that there's a huge prerequisite which is general broad knowledge about the world.

A long ago, I once fell victim for a scam when I visited SE Asia for the first time. A pleasant man on the street introduced himself as a school teacher, showed me around, then put me in a tuktuk which showed me around some more before dropping me off in front of a tailor shop. Some more work inside of the shop, a complimentary bottle of water, and they had my $400 for a bespoke coat that I would never have bought otherwise. Definitely a teaching experience. This art is also how you'd prime an LLM to produce the output you want.

Surely, large amounts of other atheist nerds must fall for these types of scams every year, where a stereotypical christian might spit on the guy and shoo him away.

I'm not saying that being religious would not increase one's chances of being susceptible, I just think that any idea will ring "true" in your head if you have zero counterfactual priors against it or if you're primed to not retrieve them from memory. That last part is the essence of what critical thinking actually is, in my opinion, and it doesn't work if you lack the knowledge. Knowing that you don't know something is also a decent alternative to having the counter-facts when you're familiar with an adjacent domain.

[infecto]

Thanks for responding and I hope my question was not read the wrong way. Genuinely curious the potential differences in folks.

[rgovostes]

Out of curiosity, was it James Tailor in Bangkok? I was whisked there on my last day by my hired guide while she stopped for an “errand”. It struck me as a preposterous hustle, but now I’m curious if this is a common ploy.

[kaivi]

It was Royal Boss Taylor, I still have it saved on Google Maps. There are a lot of these tailors, but an innumerable amount of other scams too.

[setsewerd]

Not the parent commenter, but this scam is super common in South Asia in general. It was attempted on me a couple times in India, but luckily (and in some ways, unfortunately) by that point I'd seen such a wide range of scams there that my shields were always up against potential scams.

[neom]

Not op but for me, not at all, don't care much for religion... "Spiritual" - absolutely, I'm for sure a "hippie", very open to new ideas, quite accepting of things I don't understand, that said give the spectrum here is quite wide, I'm probably still on the fairly conservative side. I've never fallen for a scam, can spot them a mile away etc.

[rogerkirkness]

I would research teleological thinking, some people's brains have larger regions associated with teleological thinking than others.

[shaky-carrousel]

Imagine what a well directed sycophancy would do in the voter base. You could make them do whatever you want, and they will be happy to do so.