[tptacek]

None. A Firecracker micro-vm is just a very small, very quick-to-start-up VM. It uses KVM, eliminates the BIOS, and implements only the minimal devices needed to boot and run server Linux. Amazon built the project for Lambda and Fargate. More about it here:

https://fly.io/blog/sandboxing-and-workload-isolation/

[atonse]

Is this sort of like what MS is doing with Windows Subsystem for Linux, where they're able to "boot" that Linux in mere seconds?

By the way, as an elixir developer Fly.io looks extremely cool. But my (mostly public sector) customers want to hear something similar to the words "AWS" when asked about hosting – so is it running on top of AWS or Azure or GCP? (instances look like they may be GCP, which is fine too).

[tptacek]

It runs on our own hardware. There's no AWS or GCP beneath it.

[atonse]

Ok thanks – this has been one of the rare things that some of those clients seem to care about (whether they're right or wrong, or rather, more conservative).

I had another question – this seems similar to what Hashicorp is doing with Boundary. Have you looked at Boundary and how this potentially compares with that, from an architecture standpoint? Of course there are parts of this that are bespoke to your infrastructure, but I'm just more curious from a nerdy-aspect of it because we're evaluating boundary as a replacement to our current setup (Wireguard bastion host), for all the other benefits like auth and logging.

[tptacek]

There is a lot of Hashi in our stack already; we orchestrate with Nomad (we have our own Firecracker task driver), we backend our certificate system --- which is awesome, certificates just work for Fly apps --- with Vault, and we use a lot of Consul.

I think our take on end-user access management is lower-level than what Boundary is trying to do. Boundary, as I understand it, sees the world the way an IdP RP does, mostly in terms of bearer tokens. We see stuff as infrastructure; a static configuration on an EC2 instance or a CI container; "just Unix". If we weren't building a PAAS, we'd probably lean much more strongly towards Boundary's way of looking at things.

As well, we care about minimizing and understanding as much of the code we expose as possible. For all the talking I've done about SSH here, the serverside of this feature is just a couple hundred lines of code; it is dwarfed by the clientside code. I couldn't say that about a Hashi product. (HashiCorp could though!)

[jefferai]

Boundary will indeed give you a bearer token after authentication to present to take further actions in its API. So you'd authenticate to Boundary, get a bearer token, and use that to make one or more requests to connect to infrastructure or services. That bit is pretty much in line with most clouds, platforms, services, etc.

However, once authorized, the actual session uses a TLS stack generated for that individual session to establish a secure tunnel. It's explained at https://www.boundaryproject.io/docs/concepts/security/connec... if you're interested.

As for complexity, while Boundary overall is by no means a couple hundred lines, I will simply say that the vast majority of code (nearly the entire API) is related to user and resource management...how users are defined and authenticated, how infrastructure and services are described for access, RBAC, etc. The actual networking code performing the secure proxying is quite minimal because at least for the TCP tunnel it's more or less specifying the acceptable TLS parameters for that session and from there you're mostly in `io.Copy` land... it probably works out to a couple hundred lines :-D

What you're doing in Fly looks super cool and the stuff you're doing on gVisor (including the user-mode Wireguard stuff) is super cool too! Thanks for writing it up. And it sounds like the two solutions are more complementary than competing, so maybe at some point in the future you'll find that Boundary has a niche to fill in your setup as well!

[atonse]

Thanks. I actually feel like if Boundary had an experience more like Tailscale does, but on top of their stack (wireguard network, secrets in vault, server/service discovery in consul, etc), that would be really powerful and a no-brainer for those of us who also use a lot of Hashicorp products.

But I'm still trying to fully understand what they're doing with Boundary. The abstractions just feel a bit off to me unlike other Hashicorp products (it's odd to me that you have to tell boundary to treat a database connection differently rather than just giving me any TCP or UDP access).

But their team does great work and has elegant designs so I trust it's more likely that the lightbulb just hasn't gone off in my head yet with Boundary.

[jefferai]

TCP targets (the only kind currently in Boundary) actually work with _any_ TCP connection. The `boundary connect <subcommand>` bits are just some CLI syntactic sugar around the main `boundary connect` command -- which works all by itself! -- that fill in IP/port and other information for various clients. (This has been an unfortunate bit of confusion for many people trying Boundary out. We'll take another look at how we can make this more obvious in our getting started/learn docs.)

It's all TCP though. Eventually we'll do more interesting things with specific protocol types.

[malnick]

Also adding to @jefferai's comment, we are working on many "better together" stories that include Vault, Consul, and Nomad. Product's like Boundary have a very large surface area in terms of potential features and road map items. There's a lot of great stuff we're currently working, stay tuned!