[chrisjj]

> The "content update" that CrowdStrike sent out was full of zeroes. Nothing else. Obviously not the intended content. And this simple data caused the driver to crash

CrowdStrike has stated that no the crash was not related to the file of zeros.

[svantex]

Interesting, do you have a link to this statement? Also, do they state what did cause the crash? At least removing the file of zeroes does solve the problem, as the instructions both from Microsoft and CrowdStrike states "Boot into safe mode. Delete C-00000291*.sys." That's the file(s) with the zeroes... See https://www.crowdstrike.com/falcon-content-update-remediatio... and https://www.youtube.com/watch?v=Bn5eRUaMZXk (3 minutes 20 seconds in).

[gruez]

AFAIK in one of the older crowdstrike threads, there was a tweet that said the driver checked for a sentinel value of AAAAA... before loading it, so an entirely blank value wouldn't have caused the issue. I can't find the source now, but some comments do seem to corroborate it:

https://news.ycombinator.com/item?id=41005546

[chrisjj]

Yes I do. https://www.crowdstrike.com/blog/falcon-update-for-windows-h...

> CrowdStrike states "Boot into safe mode. Delete C-00000291*.sys." That's the file(s) with the zeroes

That's potentially multiple files, but do we know only one comprises just zeros?

[svantex]

Right, they write rather cryptically "This is not related to null bytes contained within Channel File 291 or any other Channel File."

That's not quite the same as saying "This is not related to Channel File 291 containing all nul bytes."...

I don't have first to hand knowledge here, but rely on Dave Plummer's statement.

Regardless of zeroes or single files or not, the fact is that bad data in C-00000291.sys in combination with bad validition in the driver causes it to crash. Deleting C-00000291.sys causes the driver to stop crashing.

Anyway, my main point isn't really about this. It's about the big bang global roll out simultaneously to at least 8.5 million systems in one go that's irresponsible.

The driver architecture is the lesser evil here, although it's bad enough!

[FreakLegion]

> the fact is that bad data in C-00000291.sys in combination with bad validition in the driver causes it to crash

This is, in fact, not a fact. We really don't know yet.

CrowdStrike blue screened one of my laptops twice right as the incident was getting started, before a fix was available. There was no boot loop in my case. I was back up and in the middle of an episode of Breaking Bad the second time it got me, 30 minutes after the first. Did the agent wait that long to load a content update it had already loaded before? Maybe, but it's at least as likely that the content was loaded the whole time, and that some activity pattern set it off. Thus, I'm skeptical of the problem being simple content validation.

[chrisjj]

> the fact is that bad data in C-00000291.sys in combination with bad validition in the driver causes it to crash.

I think we've seen no evidence that data is to blame.

> Deleting C-00000291.sys causes the driver to stop crashing.

So perhaps just its existence is to blame.

> The driver architecture is the lesser evil here

Except if the crash had been limited to the driver, it would have left the machine running unprotected which is far greater an evil.

[svantex]

CrowdStrike does confirm that the data is to blame. "problematic content in Channel File 291 resulted in an out-of-bounds memory read triggering an exception" https://www.crowdstrike.com/falcon-content-update-remediatio... .

[chrisjj]

Yes - subsequent to my comment. Thanks. But how can this latest statement can be true, if the previous statement that the crash was not related to the zero bytes content is true?

[GuB-42]

Speculation: this "all zero" file is part of a signed batch, they have to have signatures, they are not that dumb (I hope...). By removing a file, the batch becomes incomplete, fails the check, and some corruption recovery mechanism takes over, most likely disabling the update and triggering an update. In the meantime, they fixed the content update, fixing the crash.