[cortesoft]

Yeah, the scope of things that an operating system needs to be able to do is basically, “all things that can be done on a computer”, so if you are trying to write full regression tests you are never going to hit all the possible combinations.

[yjftsjthsd-h]

It's not even just that, but also that a lot of the things OSs do are painful to artificially test because they're on the edge of hardware and software, or involve building the abstractions that let other software run without worrying about those details. How do you make a CI job that tests that mdevd correctly handles enumerating devices and setting their /dev nodes correctly, when the edge cases are finicky hardware devices and nondeterministic enumeration?

[pmarreck]

Does mdevd not send a signal or command out, and get one back? If so, the hardware behavior can be simulated, as can timing issues.

[notacoward]

You can test the response to a signal that way, but testing whether it was sent when it should have been can get arbitrarily hairy. I've worked places where we had to do things like rig up relays to push physical buttons (or in one case physically pull a cable) to do this level of testing. An adjacent team (in 1994!) had an actual robot that would rove over the a disk-controller circuit board delivering small electric shocks to get the same kind of test coverage on the other end of the SCSI bus. When your code has to deal with devices, which can misbehave in arbitrary ways from the voltage levels on single electrical pulses up to complex protocol violations, simulating purely-software behaviors doesn't cut it. Maybe preach about the cost/benefit ratio of different kinds of testing after you've had to literally build the tests out in the physical world.

[pmarreck]

So because you see testing all the things together as too prohibitive, you balk at testing the individual bits in isolation (hence "UNIT" testing)?