[pron]

> I'm sorry, but how you can consider anything but deductive proof as a proof?

Because if the logic is sound and something is true in the model, it cannot be disproven in the logic (and arguably engineers care more about the model, anyway). In other words, you can rely on model-theoretic proofs, which are not deductive (at least not in the logic itself).

Also, I don't think that engineers are necessarily interested in proof, but rather in verification, i.e. knowing that the proposition is true with some probability. This is also more easily achieved in the model theory.

> is nothing more than a way to say "you can choose to proof or to do some handwaving instead".

No, it isn't. You can choose to model-check (a model-theoretic proof), or you can verify with high but less than absolute certainty. Neither of these is "some handwaiving", and both, BTW, are bigger topics of research in the formal methods community than deductive proofs (take a look at formal methods conferences). And don't forget that because engineers are interested in the correctness of a system, not of an algorithm, there is always a probability that the system would fail, because there is always a probability that the hardware would fail. So below a certain probability, proving algorithm correctness is of no interest to engineers.

It also seems to me that you believe that the ability to write proofs is a choice. This is something that only people who have not used formal methods could believe. In practice, we could not write deductive proofs even if we really wanted to, because it is just too laborious. The largest software ever verified end-to-end using deductive proofs was 1/5 the size of jQuery, and it took world-experts in verification over 10 person-years (it took them 20-30, but they say they can cut it down), and it required an extreme simplification of the algorithms used. So the reality is that we have no idea how to verify programs that aren't tiny using deductive proofs, and that's why most of formal methods research is looking in other directions.

If verifying programs with deductive proofs were an actually viable option, I would be very much in favor. I'm not because it isn't, but there are other verification methods that have so far shown more promise in practice. It's not a matter of principle, but of practical feasibility.

[permeakra]

>Also, I don't think that engineers are necessarily interested in proof,

Depends on the area the engineer works in. Starting from some cost of failure you actually want a proper proof, because the inherent chance of error in incomplete induction becomes unacceptable.

>Because if the logic is sound and something is true in the model,

You can perfectly formalize this model using dependent types and use it there, don't you?

> So the reality is that we have no idea how to verify programs that aren't tiny

Yeah, I know, our brains are to small to properly verify everything, and we need to find ways to outsource as much as possible to machines.

Doesn't mean we need to willfully embrace blind faith in incomplete induction.

[pron]

> Depends on the area the engineer works in.

Sure, but let's say that it's a conservative estimate that 99% don't need proofs. I acknowledged in the beginning that deductive proofs have a place in some niches.

> You can perfectly formalize this model using dependent types and use it there, don't you?

The logic is already a formalization of the model. Yet finding a counterexample in the model directly is very often easier than finding a deductive proof in the logic. Some people got a Turing award for that discovery.

> Doesn't mean we need to willfully embrace blind faith in incomplete induction.

Formal methods is not "blind faith". Second, we haven't willfully given up on deductive proofs; it's just that for decades we've tried to scale them and couldn't, so out of necessity we found methods that have worked better in practice. I don't know if 200 years from now people won't be able to make deductive proofs feasible, but why not use formal methods to make our software better and cheaper now?

In any event, all I want is for people to know (what formal methods research and practice already does) that software correctness is an extremely complex topic, with severe challenges that are both theoretical and practical, and anyone who suggests that there is a known specific solution to the problem or that one of the many imperfect avenues we're exploring is "the future", doesn't know what they're talking about.

[permeakra]

>Sure, but let's say that it's a conservative estimate that 99% don't need proofs

99% don't need formal verification, they are OK with unit tests. If one is actually that concerned with an effort to perform actual verification, they most likely want an actual proof.

>The logic is already a formalization of the model.

So?

>it's just that for decades we've tried to scale them and couldn't,

For decades people didn't care for formal verification. Absolute majority of 'engineers' don't see any use in it. You, on the other hand, see little use for formal verification with dependent types. Well, forgive me for drawing a parallel here.

I want some specific guarantees that are fairly easy to express and keep track with dependent types, not some fancy offshot of Hoare logic. Why would I care for a method clearly inferior for my use cases?

==

Let's return a bit and examine one specific case again. Consider situation: one writes a generic merge-join function over sorted arrays. There, input arrays must sorted with same predicate over same, possibly virtual, key.

To express this in specification on the input data one need to

- quantify over the types of both arrays

- quantify over the type of the key

- and all the functions able to produce a key of this type from elements of array 1

- and all the functions able to produce a key of this type from elements of array 2

- quantify over the ordering predicate over values of the key type

- express that ordering holds for each pair of elements of the array -- i.e. quantify over indicies of arrays 1 and 2.

- For which one needs information on index bonds attached to arrays

I can, with some effort, express it in the language of type theory. I'm very interested how one can express it using, say, first order logic that, you know, doesn't allow for quantification over predicates.

[pron]

> 99% don't need formal verification, they are OK with unit tests.

Formal verification can be cheaper than tests. It's not just about a higher confidence, but also about achieving similar confidence more cheaply. That's one of the reasons why deductive proofs are not so popular in formal methods research -- they're too inflexible.

> If one is actually that concerned with an effort to perform actual verification, they most likely want an actual proof.

But we don't need to hypothesize -- they don't. Almost all formal verification in industry uses model checkers and sound static analysis. If you are interested in addressing the needs of certain small niches, that's one thing, but the thrust of formal methods research is about helping as much software as possible, and so deductive proofs are not the focus, because they're well behind other methods.

> So?

So, if you have a model checker, you don't need to write a proof. The model-checker could automatically provide a model-theoretical proof for you, at the press of a button. That's how, say, avionics software is verified. That's how Intel's chips are verified. People can't afford to write proofs for that volume of software. People write proofs only when model checking fails, or to tie together model checking results. It's no one's first choice (unless they're researchers or hobbyists).

> For decades people didn't care for formal verification.

It doesn't seem like you're familiar with the field. There was a huge boost in formal methods in the 70s followed by a research winter when people over promised and underdelivered. Then, about 20 years ago, there was a resurgence with different goals. In general, research now focuses on managing cost vs. benefit, reducing certain kinds of bugs, and finding ways to make verifiation more flexible, as we've so far been unable to make end-to-end verification scale.

> Why would I care for a method clearly inferior for my use cases?

I don't understand why you think it's inferior. It is absolutely not, and it is superior where it matters most. Expressiveness is the same, so specification is the same, but verification is flexible. You can use deductive proofs, or verifiation methods that have shown more promise.

> I'm very interested how one can express it using, say, first order logic that, you know, doesn't allow for quantification over predicates.

What? Of course FOL allows quantification over predicates and functions if they are elements of the domain, as they are in say, set theory (in fact, there is an assumption at the base of modern logic, that any mathematical theory could be expressed in some first-order language). Or, you could use HOL, if you like. But again, the theory is not the problem here. It's the chosen verification tool (and BTW, formal methods practitioners normally prefer to weaken the specification power if it reduces the cost of verification). Your example, in particular, is easily expressed in FO set theory, or many kinds of many-sorted FOL, or in HOL. For example,

    ∀ K, V . ∀ f ∈ K → V . ...

is a trivial, perfectly normal, first-order logic over sets. In fact, dependent type theories like Lean's strive to be interpretable in set theory. But really, the choice theory is the least important issue with dependent types or software verification in general.

If you are, however, interested in theory (as I am), you can read my series on TLA+. It is a first-order logic that is strictly richer and more expressive than lambda-calculus-based dependent type theories (unless you deeply embed it in them, and describe your computations in monads): https://pron.github.io/tlaplus

[permeakra]

>Almost all formal verification in industry uses model checkers and sound static analysis.

But does it happen because it is the only option or the best option? Also, encoding semantics specs is hard, so proper verification of algorithmic side is usually not an option anyway.

>I don't understand why you think it's inferior.

Because it absolutely is. It forces me to use an additional tool when I don't really need to with good enough type system.

Ugh. Look, example: memory safety. For quite some time memory leak needed to be tested for. We have a set of tools for that. Than memory control discipline in C++ arise. And then we have linear types in Rust. With Rust we don't need a separate checker: everything needed to ensure memory safety is embedded into the language type system.

Similarly, with linear algebra basic constrains can be expressed in current GHC type system. However, promoting input values to type level is awkward at best, so Haskell needs a small advancement here. Again, I don't need separate model checker here, moderately advance GHC type system works just fine.

> Of course FOL allows quantification over predicates

Wiki disagrees with you. Oh, well.

>If you are, however, interested in theory (as I am), you can read my series on TLA+.

As I said, at best I might be interested in HOL Isabelle. And even that is doubtful, so far Agda and Coq appear more useful for what I'm interested in.

[pron]

> But does it happen because it is the only option or the best option?

I don't understand the difference. Model-checking scales better than deductive proofs and is far cheaper. If you have less than 5 years to write a 10 KLOC program or your program is larger, then deductive proofs won't work. Does that mean it's better or that it's the only choice?

> Also, encoding semantics specs is hard, so proper verification of algorithmic side is usually not an option anyway.

This is true for all formal methods, but here, too, most of them beat deductive proofs. With deductive proofs you have to annotate every unit (or work much harder). Model-checking (and concolic tests) can handle partial specifications. This is yet another reason why formal methods research is not focusing its efforts on deductive proofs.

> Again, I don't need separate model checker here, moderately advance GHC type system works just fine.

It doesn't, though. Memory safety is a compositional property, as is every kind of property that can be easily checked by deductive proofs (or almost compositional, requiring one or two inferences). Most correctness properties are not compositional. That deductive proofs (of propositions that aren't compositional) don't scale is not some hypothesis of mine. They've been tried for years, and they just don't. The biggest programs ever verified with them are ~10KLOC and they took years of expert work.

> Wiki disagrees with you. Oh, well.

No, it doesn't. But if you think it does, it means you don't yet know what signatures, theories, structures, interpretations and models are -- the very basics of all formal logics -- so explaining that might take a while (in short, if propositions and functions are part of the structure, of course FOL can quantify over them; if it couldn't, it wouldn't have been the lingua franca of mathematics and the crowning achievement of formal logic). You can read my very short introduction to formal logic here: https://pron.github.io/posts/tlaplus_part2#what-is-a-logic . It's relevant to any formal logic you may want to learn at some point, as they're all more similar to one another than differen, but it may be too terse if you don't have any prior background.

> As I said, at best I might be interested in HOL Isabelle. And even that is doubtful, so far Agda and Coq appear more useful for what I'm interested in.

Sure. In general, Lean/Coq/HOL are more geared toward research -- inventing new logics, formalizing "high" mathematics -- and TLA+ is more geared toward industry use and verification of software and hardware, and because that's what I need, that's the one I use most. But it really doesn't matter which you learn -- they're all much more similar to one another than different. Their most important differences are in user-friendliness and their tooling. I would recommend Lean over Coq and Agda, though; I think it subsumes both, and its documentation and tutorials are better, so it's easier to learn. I enjoyed TLA+ and Lean more than Coq and HOL, and I haven't tried Agda. But this could be a matter of personal preference. Again, they're all more similar than different.

[dwohnitmok]

Wow. Did not anticipate such a long reply thread. Man it looks like I really should write up a comparison of my (amateur) experiences with TLA+ (TLC + TLAPS), Dafny, Idris, and Liquid Haskell.

Also permeakra, regardless of whether you agree with the points that pron is making here, if you ever find yourself interested in TLA+, I would highly recommend his four part TLA+ series (potentially as an intermediate step after e.g. Lamport's video course). It's absolutely fabulous.

[permeakra]

> if you ever find yourself interested in TLA+

Unlikely. Though I might be interested in HOL Isabelle eventually.