[narnarpapadaddy]

And with respect to function behavior, I’d view it through the lens of cyclomatic complexity.

Do I need 5-20 non-trivial test cases to cover the range of inputs this function accepts?

If yes, function is probably about the right level of behavioral complexity to add value and not overhead.

If I need only 1 test or if I need 200 tests it’s probably doing too much or too little.

[kragen]

That's not what cyclomatic complexity is, and if you think 5–20 test cases is enough for sin(), open(), or Lisp EVAL, you need your head examined.

[narnarpapadaddy]

You’re right, I suggested two different dimensions of complexity there as a lens into how much complexity a function contains. But I think the principle holds for either dimension.

I don’t think you need only 20 test cases for open(). Sometimes, more than 20 is valid because you’re saving across some other dimension of complexity. That happens and I don’t dispute it.

But the fact that you need >20 raises the question: is open() a good API?

I’m not making any particular judgment about open(), but what constitutes a good file API is hotly contested. So, for me, that example is validation of the principle: here’s an API that’s behaviorally complex and disputed. That’s exactly what I’m suggesting would happen.

Does that help clarify?

[kragen]

Yes, open() is a good API. I can't believe you're asking that question! It's close to the Platonic ideal of a good API; not that it couldn't have been designed better, but almost no interface in the software world comes close to providing as much functionality with as little interface complexity, or serving so many different callers or so many different callees. Maybe TCP/IP, HTTP, JSON, and SQL compete along some of these axes, but not much else.

No, 20 test cases is not enough for open(). It's not even close. There are 36 error cases for open() listed in the Linux man page for it.

What constitutes a good file API is not hotly contested. It was hotly contested 50 years ago; for example, the FCB-based record I/O in CP/M and MS-DOS 1.0, TOPS-20's JFN-based interface, and OS/370's various access methods for datasets were all quite different from open() and from each other. Since about 35 years ago, every new system just copies the Unix API with minor variations. Sometimes they don't use bitwise flags, for example, or their open() reports errors via additional return values or exceptions instead of an invalid file descriptor. Sometimes they have opaque file descriptor objects instead of using integers. Sometimes the filename syntax permits drive letters, stream identifiers, or variables. But nothing looks like the I/O API of Guardian, CP/M, Multics, or VAX/VMS RMS, and for good reason.