[idle_zealot]

> As a species we are so not ready for this tech.

I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access. What's killing us is MBAs and Salesmen. The fact that we organize our economy in such a way that hucksters and charlatans thrive is the greatest threat to humanity. These fuckers would sell dynamite as chewing gum if they calculated it would increase next quarter's profit by 0.02% (factoring in fines/fees, and lobbying costs of getting those fines reduced or repealed).

[jjani]

> What's killing us is MBAs and Salesmen.

SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.

[drakythe]

People are what they do. Pretty sure the man has been a Salesman ever since he started seeking funding for OpenAI.

[evmaki]

So dropping out of CS to start selling something was more important to him than 2 more years of CS education. Maybe he realized that continuing his engineering education was unnecessary because he preferred selling things. Sounds like a salesman.

[queenkjuul]

So he's a salesman

[insane_dreamer]

he's just a salesman without an MBA, which is maybe even worse

[dingnuts]

oh but he is a salesman

[LamerBeeI]

that's a dumb distinction really

[goalieca]

> On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with.

You nailed it here.

From the LLM generated search result asking how an LLM works. But this is not what the end users are being told.

>They learn to predict the next word in a sentence based on the context of the preceding words, allowing them to generate coherent and contextually relevant text

My worry is now the users are being entrapped. The LLM is "telling" them something insane and now they're being reported to the police.

[ben_w]

> If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access.

I'm now thinking of all of the times people here of sarcastically stated "OpenAI thought GPT-2 was too dangerous to release"*, as if danger only comes immediately and severely or not at all.

* wasn't even what OpenAI said, they just proposed setting a norm of caution because a release can't be undone

[dingnuts]

they meant that to be interpreted as a statement of how good GPT-2 is but the real problem is how they've marketed everything that came after to people who can't know better

it's EXACTLY the same situation as Musk selling "Full Self Driving" and then playing dumb when people go to sleep behind the wheel of their Tesla

These rich men should be in prison for this kind of false advertising

[ben_w]

> they meant that to be interpreted as a statement of how good GPT-2 is

Every previous time someone says something like this, I've looked at the original blog post again just to make sure I didn't miss something.

OpenAI's own words for GPT-2 never read like a boast to me. Caution, not a boast.

News may have bigged it up, but news does that for everything.

I mean just compare it to the GPT-5 release: they're not shy or subtle when they actually want to boast.

[jononor]

Note that OpenAI is not the same company now compared to then. All the cautious and responsible people have left or been forced out.

[ben_w]

Sure, yes.

I'm just frustrated that "doing it right" gets people laughed at both at the time and for several years later, right up until the general dangers they were publicly concerned about manifest, and then people complain about people not doing something to prevent the outcome that they were previously laughed at for attempting to prevent.

Those same cautious people who left one way or another, they were the ones most directly mocked for daring to consider the possibility their new thing might be risky and caution might be wise.

[queenkjuul]

Well they didn't really try that hard to prevent it, clearly

[Terr_]

> I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with.

Analogy: I'm not scared and surprised to hear some consumers are dangerously allergic to peanuts... However I am flabbergasted that there's multi-billion dollar industry somehow selling frickin' Peanut Immortality Panacea Serum and way too many people believe in it.

[jononor]

Haven't you heard? All the other foodsources are going to dissappear. Better get good at eating peanuts before it is too late!

[dostick]

What’s killing us is capitalism, an ill system that was predicted to fail 140 years ago but every country picked it up and tried regardless, and it failed in all 100+ experiments over 100 years. It’s always end up in inequality and suffering because it amplifies greed, and simply does not accept human being for what they are, your value being value of your learned occupation. What we end up with about is 50% of economy is useless “bullshit jobs”that are there just to employ people because capitalism can’t work otherwise!

Did you notice that every science, from physics to biology has scientists making discoveries and pushing ahead. Yet nobody ever tried to update and design new political economic system (not socialism/communism). As if capitalism was given by good and should not be questioned.

Together with ancient and not updated according to modern science democracy, that was designed for one representative for 100 people in Ancient Greece, when they knew only feudalism and so invented electable kings. And now we have one representative for 100,000 yet nobody checked if it supposed to work like that. Instead of flawed meat computers representing people we should have switched to actual computers long ago.

[saubeidl]

> What's killing us is MBAs and Salesmen

Almost. Just take it like an inch further.

What's really killing us is capitalism. MBAs and Salesmen are just the personification.

[idle_zealot]

I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas.

People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpopular. The ideas are the important part though, and they're pretty popular.

[throwup238]

You don’t need to jump to political extremes and it doesn’t help when discussing capitalism as the successor to mercantilism rather than its own political system. The answer to the problem of capitalism isn’t necessarily communism or socialism or a wholesale change of political system but lots of small solutions like well regulated markets with strong labor laws and trust busting, a more robust safety net, increased anticorruption and white collar crime enforcement, better consumer protection, and so on. Ones that can be mostly implemented regardless of the political system, given the political will.

[idle_zealot]

We had those tweaks. They got pulled back because they're inconvenient for capital owners. The fundamental problem is one of class divide: if you can draw a line between people helped by a policy and those harmed by it, and and power is concentrated on one of those sides, then they'll get their way eventually. Money is power. Therefore, if there's a policy that people with more money prefer then eventually that policy will be law. No amount of "keeping money out of politics" can get around this. You don't have to call it socialism if you don't want to, but if you want to be able to run a democracy that operates to the benefit of its people then the only way to make that stable is to enact whatever policies you need to ensure that a wealth gap and subsequent class divide doesn't form. A simple way to do that with minimal changes is to enact steep redistributive taxes that effectively cap individual wealth, and therefore individual influence gained through the market. An individual can still have power to change the world, but only through the democratic process. You still have money and a profit motive, but it can't get blown out to extremes. Basically put the economy through a sigmoid function. Market signals still work, but get weaker at high values, resulting in less extreme fluctuations and concentration of wealth.

[const_cast]

I don't even think we necessarily need socialism.

So far, we've been pretty good at identifying where capitalism just doesn't work and then bandaiding that area with legislation.

For example, capitalism has no solution for disability. In a capitalist system, everyone must work. Those who don't work should, then, be filtered out - die.

But that's obviously bad, so bandaid - SSI. We say, if you're disabled, we'll give you little socialism, so you don't die. We'll put a communal responsibility on keeping you alive because that's what's most beneficial for society.

There's no rule anywhere saying we have to just let AI make the world a worse place. No, WE decide that, even in a capitalist system.

[krapp]

You say we don't need socialism then give an example of where we need socialism.

Socialist activism is the reason we have labor rights and a minimum wage and eight hour workdays. Socialists are the reason American companies no longer field private armies to shoot striking workers dead. Socialists are the reason American schoolchildren get free lunches. Socialists died to make life in the US something more than grist for the mill for anyone who wasn't rich.

I'm going to say yes, we need socialism. And we need to admit that we need socialism. And we need to stop acting like socialism is a bad word.

[const_cast]

I mean, I agree - it's just a very tough sell. A much easier sell is demonstrating areas where we already use communal reasoning to supplement the failures of capitalism.

[sangnoir]

The best time to have moved the Overton window was decades ago. The second best time is now. It's also relevant to this age, as the current strain of capitalism is showing its ass, and everyone can see it.

[saubeidl]

I think one core issue in capitalism is that it's really hard to decide things that go against the interest of capital. See Big Oil, but also Big Tech these days.

Sure, theoretically a democratic system would allow us to make all sorts of changes to curtail the worst excesses.

In practice though, once the capitalist class has accumulated enough power, nothing that goes against their interest actually happens. They buy off politicians. They manipulate the public, be it through ad campaigns or even just straight up buying the media (Bezos buying WaPo, Musk buying Twitter).

Capital distorts democratic decision making until it becomes completely unviable to curtail their power.

[groby_b]

It's not that the terms are unpopular, it's that every system that doesn't have strong capitalist roots has lost out to more capitalist systems.

Nothing wrong with alternatives, but for that we need to let go of "let's make it just like the thing that also fails". We also need to acknowledge that socialism assumes that humans are fundamentally good, and that ignores that many are fundamentally not. We need to acknowledge that the core idea of socialism, the common good, is ill-defined. (Just like, while we're on AI, ideas of alignment on one value system suffer from that ill definition)

So, no, you don't get voted to hell for saying that the problem is capitalism. You do get downvoted for sloppy thinking if you simply propose "socialism will save us" because it's long established that in this form, the statement is not true.

Ultimately, the issue is that HN is not made for that discussion. You'd need nuanced political debate, and the internet does not seem conducive for that. That's the second reason for downvotes - HN is not interested in contentious political debate, for better or worse.

But no, it's not because we just all immediately think of the USSR/Maoist China.

[dragonwriter]

> It's not that the terms are unpopular, it's that every system that doesn't have strong capitalist roots has lost out to more capitalist systems.

The system with the strongest capitalist roots (i.e., the dominant system of the industrialized West of the mid-19th century which is the system for which “capitalism” as a term was coined to describe) progressively lost out over the time since that name was coined for it by its socialist critics to a more socialist system, the modern mixed economy, through changes largely driven by socialist critics.

> We also need to acknowledge that socialism assumes that humans are fundamentally good.

Socialism does not assume this. I would even argue that the idea that “humans are fundamentally good” is even a coherent claim that can be right or wrong requires a concept of a particular kind of external mrorality that is difficult to reconcile with the premises of socialism.

> We need to acknowledge that the core idea of socialism, the common good, is ill-defined.

The core idea of socialism (like that of democracy in the political sphere, because socialism is exactly democracy without an artificial divide between political and economic spheres) is less “the common good” as it is “the common good is ill-defined, while the interests of individuals are known to the individual more than any third party, and fairness requires equal empowerment of individuals to pursue their interests.”

[sangnoir]

> It's not that the terms are unpopular, it's that every system that doesn't have strong capitalist roots has lost out to more capitalist systems.

The arch of history is long, and we are naturally biased to think of the present as the culmination of history - but it's just a point in time. I do not think "socialism will save us" - but I believe there is a breaking point where society simply will not accept a - as you put it - "more capitalist system".

Politics and economic systems go hand in hand, any economic system, practiced in extremis will be destabilizing. I posit that theoretically the "more capitalist system" wins over the less capitalist one, up to a point, where winning comes at the cost of killing the host society, and thus itself. This is simply a thought experiment, I am not making any declarations on where the US is on this axis.

[dingnuts]

those stupid ignorant Americans who associate the United Soviet Socialist Republic with socialism. where did they ever get the idea that socialism has ever been tried???

don't worry, I associate Marxism with REAL socialist regimes, like Cambodia

[saubeidl]

Just like we associate the Democratic Peoples Republic of North Korea with democracy and republicanism :-)

[dragonwriter]

> those stupid ignorant Americans who associate the United Soviet Socialist Republic with socialism

It was the Union of Soviet Socialist Republics, not the United Soviet Socialist Republic, and if you believe that ideological faction and their naming of governments, I want to see your face when you learn about the German Democratic Republic and the Democratic People’s Republic of Korea.

[I_dream_of_Geni]

Pssst: or the DRC

[saubeidl]

I find that part of being intellectually curious and a good engineer is questioning systems.

If we had a systems architecture in an engineering project that kept leading to bad results, we would consider rearchitecting how said system is run.

I believe it is important to make it clear to people that capitalism, too, is just a (imo deeply flawed) systems design. It is not a natural constant. If we find it leads to negative outcomes (which I strongly believe), we can and should talk about rearchitecting.

[kiba]

What people describe capitalism is really just monopolies. Capitalism meanwhile is an utterly useless label. People have their own conception of what capitalism mean.

Monopolies are connected to the housing crisis as to privileges like copyright. They're unwarranted or undertaxed government granted privileges.

[mallowdram]

The tech was never ready, it's built on faulty units. Pull the plug.

[throwaway290]

This is beyond naive.

who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech

and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do you think he also was misled by adverts or something? or big corps won't buy the patent and run with it commercially advertising it like they do anyway? or if you pretend money and big corps don't exist and we deploy it for free for friends, same problem? etc

if you went back in time and killed capitalism in US and switched to command economy where people innovate because they will be sent to gulag otherwise, for sure most of today tech including this would not be developed. but it seems like a pointless exercise.

instead what should happen is all these megacorps are sued for copyright infringement, fined and shut down. the model won't be smart enough to sweet talk ppl into suicide if megacorps can't infringe our copyrights to train it.

[LamerBeeI]

you're making this comment on the world wide web, which was invented and given away for free - likely against a linux machine, an open and free project

[throwaway290]

you are making this comment on computer hardware which is a prerequisite for www and which was invented and made because it's profitable

[tedivm]

That's a bit of a stretch. Lots of stuff that gets published still has patents associated with it. Just because something is done in the open doesn't mean it can't be commercialized.

[idle_zealot]

Eh, we fund a lot of research through grants. It may not have been 2022, but a society without as strong of a profit motive would have discovered transformer models and LLMs eventually. They probably wouldn't have scaled up datacenter production and tried to shove a chatbox into every digital interface though.

> and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results

Yes, my point is exactly that I don't think the results would be the same if people were running it themselves at home, without the marketing machine running full bore to convince the world that these impressive toys are machine oracles. You'd end up with a handful of crazies, that's unavoidable, but it wouldn't be a civilizational threat.

[saubeidl]

You just perfectly described why capitalism is the problem.