[jorangreef]

Was this a question or a statement? :) (I appreciate both, let me add!)

But sincerely, I think we don’t share the same philosophy:

End to end correctness of software is a systems design problem, not a language problem.

I believe that if you try to shoehorn too much safety into a language (100% vs 90%) you get into trouble, and don’t solve safety end to end, like we try to do in TigerBeetle, systematically through TigerStyle, Deterministic Simulation Testing of the system as a whole etc.

I try to write about much of this in the post, to share our ideas, but it comes down to power-to-weight ratio, Zig’s essential simplicity, and recognizing (and coming to terms with) the fact that:

No language can solve end to end systems safety. For this you need systems thinking.

[neckbeards]

It's a bit of both! I’d love feedback from someone more experienced with Zig, especially if there’s any flaw in my reasoning here.

The programming language is part of the system design. The abstractions, invariants and guarantees the language provides define what classes of bugs are even possible to have. For example, Rust’s ownership and lifetime semantics eliminate entire categories of memory and concurrency errors that would otherwise surface as “system-design” issues in C or Zig.

When you say “power to weight ratio”, could you elaborate on how that applies relative to C++ in the context of TigerBeetle? You mentioned io_uring support being added. What makes Zig uniquely suited for that compared to a more mature language like C++, which already offers a concurrency model and a sophisticated type system you can selectively use?

You also mentioned prefetch support. That's a lot easier to implement in other languages. I’m curious what specifically made Zig the better fit for these optimizations in your experience.

I appreciate you taking the time to respond to my question.

[jorangreef]

Glad to hear!

Let me invert our roles!

What are 3 of some of the hardest correctness problems in TigerBeetle—and how does TigerBeetle solve them?

Hint: None of these would be solved by language.

[neckbeards]

I don't know. I don’t doubt that TigerBeetle’s hardest correctness challenges can’t be solved by language alone. TigerBeetle has impressive guarantees! Those are inherently systems-design problems.

But the language defines the failure surface and the cost of getting those systems problems right. Likewise, the difficulty and performance of implementing correctness mechanisms (lock-free data structures, concurrent logs or safe async I/O) are directly shaped by the language’s semantics and toolchain.

So while I agree end-to-end correctness is a systems-design problem, the language choice determines how much of that correctness is enforceable, how much is manual and how hard it is to make the right design fast and reliable.

[jorangreef]

More precisely, “TigerBeetle’s hardest correctness challenges can’t be solved by language AT ALL”.

The “harder problems” in TigerBeetle then (and where we invested millions literally in engineering):

- distributed system strict serializability (cf. our Jepsen audit)

- storage fault safety (TB expects disks to write/read to/from the wrong location, or drop writes entirely, and MUST survive this, all while preserving strict serializability and maximizing availability given the redundancy at hand—it’s one of the first databases in the world that can solve this to these tolerances, cf. “Protocol-Aware Recovery for Consensus-Based Storage”)

I will leave the third one to you! :)

But C++ (or even writing TB in a memory safe language like JavaScript—to reduce the point to absurdity) would have done nothing to solve the hundreds of distributed system bugs our simulators (and TigerStyle methodology) find, ~none of which are language bugs.

Again:

Distributed systems end to end correctness is a systems design problem, not a language problem.

What do we want from our language then?

Power to weight ratio, literal power over the metal (e.g. no OOM or hidden allocations or copies), with essential simplicity in syntax, and explicitness in everything.

I know of no better language for TigerBeetle in this than Zig.

But to make the point more strongly, I don’t think we would have succeeded as a project and company if we hadn’t picked Zig.

C/C++/Rust I feel almost certainly would have cost more, but also would not have given the same quality in terms of design and what I wanted to explore in TigerStyle. It simply would not have been TigerBeetle. We truly needed Zig to exist, and I’m glad it did at the time we needed it.

[neckbeards]

Ok, thanks! Appreciate your time and it’s awesome to see the TigerBeetle team solving hard problems in OLTP while giving back.