[colechristensen]

I was able to identify, diagnose, fix, and upstream a minor bug in and erlang/OTP ssh key implementation with Opus in maybe 20 minutes (+2 weeks or so for upstream). It is not impossible that I could have done this before, but it would have taken days or weeks. The actual fix was about 2 lines of code, hardly AI slop, but getting there would have been quite the slog, and I never would have done it.

There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.

[medlazik]

I said absolutely nothing about LLMs, which is a fantastic tool I'm using every day. I'm talking about marketing.

[gallerdude]

So let’s say you’re in Anthropic’s shoes. You see that LLM’s are getting better and better, and it’s very possible that they will have some impact on jobs in the next few years, and a very meaningful impact on cybersecurity.

Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.

[ifwinterco]

The issue is both OpenAI and Anthropic have lied so many times that it’s no longer rational to take anything they say at face value.

Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast

[watwut]

I think that Anthropic is fully absolutely unethical. And they lied a lot. They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.

If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.

It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.

[colechristensen]

The point I'm trying to make is Anthropic's marketing about broad security risk related to the capability of its models is a valid concern though their dog and pony show really overdid it, probably to the detriment of us all for many reasons. It is indeed amplifying the abilities of people to find and exploit security issues.

The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.

The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.