[notepad0x90]

working in software, you should know how insanely complex software is, even google, amazon, microsoft, cloudflare and such have outages. mistakes happen because humans are involved. it is the nature and risk of depending complex systems. bridges by comparison are not that complicated.

I actually expected their stock to drop a lot more than this, but goes to show you how valuable they are. investors know that any dip is only temporary because no one is getting rid of crowdstrike.

Think of the security landscape as early 90's new york city at night and crowdstrike as the big bulky guy with lots of guns who protects you for a fee, if he makes a mistakes and hurts you, you will be mad but in the end your need for protection does not suddenly go away and it was a one time mistake.

[TheOtherHobbes]

In which case "Are you awake and sane?" would be a sensible reality check before heading out.

You're trying to hand-wave away the inexcusable. The outage is a symptom. The problem is the lack of even the most basic testing.

Clearly these files are sent out without even a minimal sanity check. That is a problem, and it's not something that can be hand-waved away.

[notepad0x90]

In the 3-4 decades of the security industry, testing signature files to see if they trigger a corner case system crash has never been practiced. You and others are proclaiming yourselves to be experts in an area of technology you have no experience in. This was not a software update!!

[lores]

Then that's 3-4 decades of massive incompetence, isn't it? "Testing before pushing an update" is basic engineering, they have a huge scale so huge responsibility, and they have the money to perform the tests and hire people who aren't entirely stupid. That's gross malpractice.

[notepad0x90]

testing for software, not for content. you test, and fuzz the software that processes the updates, not the content files themselves. it's like a post on HN crashing HN and you claiming HN should have tested each post before allowing it to be displayed. you test code not data, and I dare you to back up any claim that data processed by software should also be tested in the same way. Everyone is suddenly an expert in AV content updates lol.

[ciuokan]

I used to work for Microsoft in a team adjacent to the Defender team that worked on signature updates and I know for sure that these were tested before being rolled out - I saw the Azure Devops pipelines they used to do this. If other companies aren't doing this then that's their incompetence but be assured that it's not industry-wide.

[notepad0x90]

I'm not saying they don't test them, I'm saying they don't do code tests, as in unit tests and all that. I have no idea what they do, I'm just speculating here, but if in fact they do no testing at all, then I agree that would be pretty bad.I would think their testing would be for how well it detects things and/or performance impact and I'd expect it to be automated deployment (i.e.: test cases are passing = gets deployed), i guess they don't have "did the system crash" check in their pipelines? In your experience at MS, did they test for system/sensor availability impact?

[lores]

A config file IS code. And yes, even a post can theoretically break a site (SQL injection, say), so if you're pushing data to a million PCs you'd better be testing it.

[notepad0x90]

You're right, but "testing" could mean anything, you'd need to have the foresight to anticipate the config crashing the program. Is it common to test for that scenario with config files?