[batterseapower]

If you like the idea of this library you'll probably like the book "How to Measure Anything" by Douglas Hubbard (https://www.goodreads.com/book/show/20933591-how-to-measure-...). It's all about how to get sensible confidence intervals for things that are often considered unmeasurable such as the value of IT security. The book mostly uses Excel to do this modelling, but it looks like riskquant would be an excellent alternative on that approach, that for the more technically minded practitioner.

[jacques_chester]

The relevant book for this is Measuring and Managing Information Risk: A FAIR Approach by Freund and Jones[0].

Both books are worth reading; Hubbard's influence on FAIR is noticeable and positive. FAIR has the advantage that it comes with a fairly built-out ontology for assembling data or estimates. The OP touches on the top level (Loss Event Magnitude and Loss Event Frequency), but the ontology goes quite deep and can be used at multiple levels of detail.

The calculations are not difficult, I've implemented them twice in proofs-of-concept, including one that produces pretty charts.

The difficult part, to be honest, is that developing good estimates is difficult and frequently uncomfortable and the gains are not easily internalised.

Additionally, serious tool support is lacking in the places where it would make a difference -- issue trackers, for example.

[0] https://www.amazon.com/Measuring-Managing-Information-Risk-A...

edit -- Another good book in this area is Waltzing with Bears by DeMarco & Lister. A short, funny, insightful read, as you'd expect from the authors of PeopleWare: https://www.amazon.com/Waltzing-Bears-Managing-Software-Proj...

[lifeisstillgood]

I regularly put a risk / loss / impact assessment into my issue tracker tickets - it's not the tool support is not there, it's that

a) everyone else needs to do this across the board

b) it's still just a guess - normalising my guess and you're guess is hard

[mindcrime]

b) it's still just a guess - normalising my guess and you're guess is hard

True, but that's where the calibrated probability assessment stuff comes in. If you can at least establish obviously correct (if very course grained) upper and lower bounds for estimates and then gradually shrink them in until you aren't confident doing so anymore, you have something at least slightly better than a complete guess. And if you can choose the correct distribution that the actual values would vary over, then you can use Hubbard's approach using a Monte Carlo simulation, and get some insight into likely outcomes.

No, it's not a perfect approach, but it gives you a little something to hang your hat on.

[jacques_chester]

Tool support would ideally both of those problems easier. But good estimating is in itself fundamentally a "system 2" activity.

[malenos]

system 2: is that a reference to 'no silver bullet'?

[elbear]

I assume it's a reference to Kahneman's "Thinking, Fast and Slow".

[malenos]

Thanks

[thanatropism]

I have that book. Basically I’m the last in a giveaway chain and can’t honestly recommend it enough that someone should lug it home. Next time I move it’s going on the trash.

It’s really not very good, even for executives who shouldn’t care for technicalities. The best thing are the calibration exercises. But my advice is, skip this one.

[qznc]

Can you elaborate?

I'm half-way through it. I know most of the general stuff already but my knowledge is from lots of sources I mostly forgot. This books seems to be a good collection for this topic. At least, I don't know any substitute.

[thanatropism]

It overstates its claims. An admittedly unfair "take" would be that it encourages people to pin made-up numbers and take solace in having quantified the qualitative.

For example: it tells you to do Monte Carlo simulations with made-up probability distributions, but silently lets the issue of the joint distribution -- how the made-up random variables correlate. But so much risk is driven not by marginals (say, I know the physical parameters for my chair and have a certain confidence that it won't crumble like carboard) but by correlations, even apparently distant ones (there's fracking or whatever going on and destabilizing the upper layers of the Earth, increasing the chances of earthquakes).

An even broader critique is the McNamara fallacy:

> "The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide."

----

There's this book I like called "Guesstimations in the back of a napkin" or something -- it's a book of exercises in Fermi-type estimation. It encourages you to consider the whole, to think bottom-up, top-down and core-out. It preserves a keen sense of the qualitative complexity of problems even as it encourages you to, well, make wild guesses.

[mindcrime]

There's this book I like called "Guesstimations in the back of a napkin" or something

This? https://www.amazon.com/Guesstimation-Solving-Worlds-Problems...

[mindcrime]

It’s really not very good

Compared to what?

But my advice is, skip this one.

And read what instead?

Not trying to start an argument here, I'm genuinely curious, as I consider How To Measure Anything to be one of the best books I've ever read (and I read a lot of books), and I recommend it highly to, well, pretty much everybody. If you feel that there's a better resource out there that relates to these topics, I'd be curious to know about it.

[thanatropism]

I'm not very fond of Taleb, but well -- anything by Taleb.

Exercise books for Fermi estimates like Guesstimation, etc.

Further out, something in systems thinking, maybe Donella Meadows' "Thinking in systems". Further further out, maybe those Stafford Beer papers about the Viable Systems Model? At one point Beer and Allende thought they were about to implement Red Plenty.

---

I understand the businessy logic that nothing is so fundamentally qualitative that it shouldn't be quantified. But you'll always be safer if you keep rich qualitative models and treat quantification as gravy on top of that.

The extreme opposite of rich qualitative models is the Soviet method of material balances. Halfway through there's the McNamara Fallacy:

https://en.wikipedia.org/wiki/McNamara_fallacy

[mindcrime]

I'm not very fond of Taleb, but well -- anything by Taleb

Yeah, Fooled by Randomness and The Black Swan were both pretty good. I haven't necessarily thought of them as significantly overlapping with the HTMA stuff up until this point, but now that you mention it I can see a connection. I should probably go back and re-read both, and read Antifragile.

maybe those Stafford Beer papers about the Viable Systems Model?

Hmm... never heard of "Viable Systems Model" before, so I'll have to go read up on that. Thanks for the pointer.

Exercise books for Fermi estimates like Guesstimation, etc

I'll take a look at Guesstimation. Thanks for the pointer on that as well.

But you'll always be safer if you keep rich qualitative models and treat quantification as gravy on top of that.

I can buy that. I'm a fan of using approaches like Hubbard's to quantify things to a point. I do think his approach can supply a bit of extra rigor and some useful bounds to things that otherwise seem impossible to quantify at all. But it's not a perfect system by any means. The two biggest risks, so far as I can tell, would be leaving a variable (or more than one) out of your model completely, or using the wrong probability distributions for the various variables when doing the simulation part.