[Jtsummers]

I was working on something similar to say, stepped away, and saw your post. So I'll add to it:

TLA+'s place is as part of your design process (you have one, right?). It's the thing you do before committing any code to the project or radically restructuring an existing project (prototype code fits in the design process, but it shouldn't be considered committed most of the time). By spending a bit of time thinking about the design, and using tools like TLA+ to validate that design, you can save yourself months and years of heartache down the road.

By way of anecdote, some colleagues were working on a concurrent algorithm, needing to share data between two embedded processors talking over a databus. A bit more time spent on the design, versus the rather slapdash approach they took, would've saved them 6 months of debugging and tweaking until it worked. The issues were subtle, so it "worked" but wasn't totally correct and tracing the errors they were seeing back to that code was non-obvious (and hard, debugging embedded isn't trivial). It was quite literally a case where skipping a couple weeks of design cost them months later.

[sagichmal]

> TLA+'s place is as part of your design process ... By spending a bit of time thinking about the design, and using tools like TLA+ to validate that design

I think you dramatically underestimate the amount of effort required to produce a formal model of any nontrivial system, and overestimate the stability of system definitions in response to changing business requirements.

[ahelwer]

Eh. I used TLA+ for a few projects inside Microsoft. It probably takes a couple of weeks to write a spec for a complicated problem. Then less time to implement it, because all the heavy lifting has already been done - in many cases you're directly transcribing the spec logic without having to think very hard.

The sort of things for which you write specs are generally "core" functionality that doesn't change very often. If people are constantly mucking around in that area without writing specs, your system will be a pile of crap.

[hwayne]

I think you dramatically underestimate the benefit of writing a formal model! I've run classes where we've managed to discover bugs in the client's actual production system with an afternoon of modeling.

[neolog]

How important is it to discover new bugs? Surely any large production system already has a backlog of tons of known ones. So if I want to fix bugs in my system, I can just go to the issue tracker and pick one.

[Jtsummers]

Depends on what "discover" means. In the sense that Hillel is speaking of here, it probably means more like: Discover the bug exists and also discover, to a very high degree of certainty if not absolute certainty, the location of the bug in the code base.

That discovery is actionable, many other bug discoveries are not without further investigation.

[hwayne]

At the level that you're usually writing specs, the bugs you discover are often very serious ones, such as dropping data.

[Jtsummers]

I'm not underestimating anything because I'm not suggesting producing a formal model of arbitrary, nontrivial systems. My critical point is: Design your system before committing any code to the end result [0]. Formal methods and models are one tool in your designer's kit.

[0] This statement should be qualified with: Especially if it's at all novel to you, your team, your organization, or the world. And the amount of time to spend varies based on its true novelty. A team that's made hundreds of RoR sites can churn a new one out in a week with minimal upfront design effort. A team that's making the next RoR should probably spend more time thinking about what that means before making, or committing code to, it.

[pron]

In practice, the effort is smaller and cheaper than finding bugs in other ways. But yes, learning where and how to best apply TLA+ takes some practice (less than learning a programming language, but more than learning a new build tool).