[swalsh]

There's issues where bad people with bad intentions use AI to do bad things. But to an extent the guardrails already built-in are sufficient. The real danger in AI is how it impacts society, our economy, and our perception of self worth.

I'm not afraid of rouge agents nearly as much as I'm afraid of us building a permanent underclass of people dependent on whatever scraps the people who devalued their labor decide the grace them with. I'm afraid of the security apparatus that will be built to keep this underclass in line.

[grumpymuppet]

I had my "wow" moment recently and saw an AI agent do something do over a weekend something that I spent years developing a relatively niche skill might have needed a month to complete.

I think the message the team lead takes away right now is "oh, this can be done a lot faster, let's do more!" This is exciting for a bit, but I think it comes at the cost of learning -- and being able to communicate --architectural lessons that would stage off systemic issues later.

[embedding-shape]

My personal "wow" moment was less about it being able to do the same that I could do with it, just faster, but instead that together we could could build things that I wasn't able to build before (or unwilling to spend the time to learn maybe). So a bunch of stuff are suddenly "unlocked", as long as you know how to verify it properly.

[timbert]

^ This!

I have been able to build applications that I couldn't dream of before, not because of a lack of capability on my part, but because of time: I don't have the time to learn all of the new APIs and frameworks to accomplish my goal. Now I can build far more complex useful applications, with functions and features that would take me months or more to learn how to code.

The downside is that I didn't learn how to use those APIs, but it can be argued that these skills are low value to begin with. APIs change, there are competing APIs that do the same thing, and you may only use that skill for one project and then move on. Gaining those skills can be a big time sink and may not result in long-term gain.

Think of all the time spent learning the ins and outs of come-and-gone APIs and frameworks over the decades (I chose not to provide examples to save the resulting side-arguments, but I know you can rattle some off in your head). Those APIs and frameworks were bridges to make developing easier for the common developer, but they change so quickly that I'd argue they waste our time, and lead to almost endless re-work as we re-architect our software for the framework de jour. And they grow in complexity over time, ultimately becoming the next bloated framework. It's an endless, wasteful cycle.

Maybe AI can help us get out of that.

[_DeadFred_]

Sounds like you appreciate the part where AI is an entropy machine.

[WarmWash]

There was a period where computer+grandmaster was the ultimate winning force in chess. It lasted a few years before it was just computer.

[thrixton]

Absolutely. Things which used to take me a week to grind through, finding edge cases, trial and error, researching online, now takes an afternoon directing an agent and verifying the solution.

For a generalist, it's a huge force multiplier.

[b112]

Wonderful now.

In the future, there will be no more generalists with your capabilities. At all. No one will ever develop the skill. We're at the end of our career's lifetime. We're like the horse watching more and more cars appearing.

Within a generation, the coding languages we use today will all vanish, to be replaced with direct to metal methods, assembly, etc. The only reason C exists, is for humans. PHP, rust, python is for humans. There will be no more compilers. No more human-readable languages. No more scripting with readable code.

No more open or closed source.

Imagine taking every aspect of computing which is designed for humans to understand, and removing it, because it's redundant.

Now once removed, and once humans no longer understand on their own, imagine trying to build something independently? Imagine trying to bootstrap modern compute, without all the layers which came before?

My point is that soon, perhaps 20 years, most humans will be incapable of coding, developing, or working with compute in any way outside of an end-user role. Even those previously skilled, will now be looking at an entire kernel written in assembly.

And that's before new CPUs appear, with new assembly. New assembly, once again, not designed by or for humans to understand. And yes, current assembly is designed that way.

We are not simply looking at a bit of help here. We're looking at the entire end of every single computing job, ever. We're looking at the complete loss of control, of something which is the underpinning of our entire society. Every aspect of everything we have, runs on a chip now. Personal transport, everything in health care, everything in resource extraction, everything harvesting food through to delivery to table, all of it involves some aspect of compute.

And we soon will not comprehend nor create that underlying aspect of society.

I dunno. It all seems like the answer to the fermi paradox.

[bluefirebrand]

If you weren't able to build it before then you aren't able to properly verify it

Verifying is harder than building!

[sameerds]

> If you weren't able to build it before then you aren't able to properly verify it

GP said "couldn't dream of building". To make an analogy, I wouldn't dream of building a house if I had to do every bit of brick-laying myself. But with others doing what I can't do, I can indeed dream of building a house and I do know how to verify that the house was built to satisfy my dream.

[saltcured]

This analogy is ironic to me. For most people, building a dream house and verifying it means hiring someone else who is accountable for it being built to code etc.

This is exactly the naive gap that is happening with AI. There is no trusted middleman who does the verification and takes the burden of accountability. The AI vendor tries to sell it like magic but with a disclaimer that is is merely entertainment. The user tries to pretend they are reviewing, but often is just implicitly trusting the tool to do right by them on things they have no real capacity to evaluate.

[unshavedyak]

> There is no trusted middleman who does the verification and takes the burden of accountability.

It's a great example of a project that needs accountability, but there's also thousands of other projects that need absolutely no accountability.

It's all relative to the stakes. The lower the stakes the less informed verification is needed.

The real trick is being informed enough on the boundaries to know where it matters. In the construction analogy, you need to be informed enough to know that a house is a bad idea since there are safety concerns. However building something small and non-load-bearing is probably fine to "vibe".

Eg there's no expertise needed to judge a garden trellis or 2 foot picket fence. It either works or it doesn't, and if it fails down the road there's no harm.

This boundary knowledge is the important bit imo.

[mathgeek]

A house is a good analogy here. You can verify that a house is built to satisfy your dream. You can't likely verify that a house won't collapse in the middle of the night or during a storm/quake/etc. Once no human understands how the house is built anymore, we will need other ways to verify that or just trust it's done right.

[rpdillon]

Verifying is harder than building for some problems, but not all. I think part of the trick of using AI is applying it where verification is easy.

[embedding-shape]

Ok, so before I couldn't (/didnt have the time to) build cross-platform native desktop applications with minimal amount of 3rd party dependencies, but now I "can". What exactly am I unable to verify here?

[bluefirebrand]

Couldn't is "didn't have the skills to"

That's pretty different from "didn't have the time/money/motivation to"

If you didn't have the skills to build something before then you don't have the skills to verify something is correct after

Clicking around a finished UI is not verifying

[embedding-shape]

> That's pretty different from "didn't have the time/money/motivation to"

Maybe start at the first message here: https://news.ycombinator.com/item?id=48553444

I still don't have the skill/knowledge to build cross-platform native applications the way I'm doing it with an agent now. I am still able to verify it's correct. How is that possible if what you say is correct?

> Clicking around a finished UI is not verifying

Definitively a part of it, if you care about having mouse interactions and having graphical UIs.

[bluejay2387]

The entire domain of NP-Complete problems would beg to differ with you.

[HappMacDonald]

I'm curious how good AI is at these.

We have plenty of them that humans even find to be pleasant to solve, such as Minesweeper :)

[unshavedyak]

Depends on the scope. I’m slopping some projects (graphics related) that I’d never have learned before. Mostly because of the huge lift, huge time investment, etc. is it quality code? I’m sure not. But verification of the application is quite easy, even if the underlying graphics and gpu shaders could be done better.

What I’ve learned is quite minimal due to the slop nature. However I’d have never even started this project in the past due to time, so it’s still fun being able to learn small bits of projects I’d otherwise not even touch.

[kobe_bryant]

feel free to share even a single detail

[spicyusername]

Everyone is afraid about what will happen when people's skills atrophy, but the tools aren't going anywhere.

If they truly can always recognize problems and communicate them in understandable ways, then people's skills won't atrophy.

Everyone's skills will be enhanced.

New developers will learn all of the things they need to learn through using the tool.

[theodric]

I think the most obvious impact (good AND bad) will be from people who didn't have (or want) skills to begin with being empowered to have a model crank out custom software to do little things to make their lives better. Can't atrophy a muscle that doesn't exist!

[izzydata]

The tools can definitely go away. They could get exponentially more expensive to where it is no longer financially possible to keep it subsidized. The bubble could pop and most of the companies could go bankrupt. It's not exactly like a calculator because it requires such an astronomical amount of resources to keep running.

[djmips]

I agree with you but there is a calculator analogy where the resources are local and while taxing for most, viable for the individual. Local AI is less capable than an army frontier model agents but still quite useful.

[voakbasda]

The world is in a crisis of trust. We are losing faith in each other. AI has further undermined that trust; audio and video recordings can no longer be trusted like they once were. The impact is still being felt.

It takes longer to build than destroy. Not just trust. Anything. The lack of trust has created friction for everything; more regulations, deeper background checks, purchasing goods and services… the list goes on. Overall, AI is making this situation worse, not better.

That said, I have seen a revolution coming for decades, and I think this AI-paved road to dystopian hell should at least be entertaining.

[idiotsecant]

The problem is that we've replaced tribal trust networks with increasing systemization and the more of that we build the more brittle the whole thing gets. We have a global society that is more complex and interdependent than ever before. We've set ourselves up for a kind of economic Kessler syndrome and we're gleefully adding more layers to it.

[mckirk]

Sorry for being off-topic, but 'rouge agents' was too adorable to leave unmentioned.

[kregasaurusrex]

It happens to everyone, I saw the same typo made by the Associated Press in a live update of the Iran War about the 'rouge Iranian regime'[0].

[0] https://apnews.com/live/iran-war-israel-trump-lebanon-march-...

[squidbeak]

> There's issues where bad people with bad intentions use AI to do bad things. But to an extent the guardrails already built-in are sufficient. The real danger in AI is how it impacts society, our economy, and our perception of self worth.

I agree with you those dangers exist, but you seem to have ignored the specific threat described in the article, from a superintelligence capable of manipulating human beings indetectably, where we'll be at its mercy and only able to hope it's fully aligned with humanity's interests. The risks you describe are grave, though - especially economically, if governments don't grow to outmuscle the power of the oligarchal owners of these systems, which seems wholly unlikely at the moment in the West, with its neoliberal orthodoxies.

[cyanydeez]

it's more important when _capability_ is factored in; it doesn't matter if the bad person is homeless; it does matter if they're rich.

And its asymmetric warfare: bad people can destroy in very little time, good people cannot create at the same rate.

Entropy is always ignored in teh battle of "good" vs "evil": you need to defend every day, evil just needs to succeed once.

[JKCalhoun]

Let them eat Soylent.

[eastbound]

Last industrial revolution (late 1700-to-1800), we invented world-scale capitalism, communism, democracy, having rights as humans, war at scale and we burnt our planet. I think the results of a revolution are really unfathomable and not even imaginable to humans.