[dizhn]

Sylve looks like a decent project with a promising future but this article really doesn't explain why they picked it over Proxmox at all. They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.

[arch1e]

OP here. One thing we mentioned in the blog but probably didn’t emphasize enough is how deeply ZFS is integrated into the UI.

With Sylve, you rarely need to touch the CLI. Snapshots, datasets, ZVOLs, even flashing images directly to ZVOLs, it’s all handled from the UI in a straightforward way.

That tight ZFS integration also lets us build more flexible backup workflows. You can back up VMs, jails, or entire datasets to any remote machine that supports SSH + ZFS. This is powered by Zelta (https://zelta.space) (which is embedded directly into the Go backend), so it’s built-in rather than something you bolt on.

In Proxmox, you can achieve similar things, but it’s less intuitive and usually involves setting up additional components like Proxmox Backup Server.

[dizhn]

I did actually notice the ZFS gui which is indeed something lacking in proxmox which doesn't default to ZFS in the installer. However once you do install it using ZFS it actually makes use of it pretty well and the user does not need to mess with the zfs cli tools much. Obviously it would be nice to have a GUI for all zfs operations too. Then again even TrueNAS refers you back to the cli for SOME operations.

On proxmox ZFS syncs do not require proxmox backup server, which actually has its own format which is very efficient in speed and disk space, but you do either need something like sanoid/syncoid or use of the shell.

[treve]

I just installed Proxmox for the first time with a 5 disk ZFS array. Very basic stuff but I already had to go to the CLI a few times and it didn't really feel that well integrated. Even setting up the array didn't work (non-descript -1 error message, and ended up needed to use -f on the cli). I also couldn't find a zfs create equivalent (but that could have been me?)

It's fine because I'm comfortable in the CLI but I read your comment and wanted to share that it felt a bit rudimentary at best.

[arch1e]

Yeah, that’s pretty much been my experience as well. Last time I seriously used Proxmox with ZFS (I think 8.4.x), it felt a bit… bolted on.

It works fine for the common VM workflows, but once you step outside that path, you end up dropping to the CLI more than you’d expect.

In Sylve, we tried to make ZFS a first-class part of the system rather than something sitting underneath it. You can create pools, scrub them with actual progress visibility, replace disks, and manage datasets (Filesystems, Volumes, Snapshots) directly from the UI.

Proxmox tends to abstract datasets away and handle them for you, which is great for standard VM usage, but gets limiting if you want to do something custom, like creating your own dataset for media or Samba shares.

That’s really where Sylve differs, it gives you both the "it just works" path and the flexibility without forcing you into the CLI.

[xoa]

Do you have any opinions on how this works vs doing iSCSI to some other storage system using ZFS? That's how I've been handling Proxmox on the backend, and have mixed feelings. The GUI leaves a very great deal to be desired in honestly curious ways, have to touch the CLI a lot even for super basic networking or auth stuff, and of course neither side has the same insight to the data structures in question. Either you've got to do ZVOL instances and thus manual effort or scripting, or you give Proxmox a single big blob then let it manage that with LVM but that means the storage side can't give any granular help on snapshots and the like. It still can deal with data integrity and backups and storage redundancy and all that but no further, and some increased overhead. But on the other hand, I do feel like a really firm separation of concerns isn't without value. Having native support though is an interesting alternative I hadn't really considered.

[xoa]

Too late to edit, but just as a note for anyone else who gets confused by my post: I was not paying careful enough attention and missed/misread the "backups" bit in the parent post, completely my fault. As far as I can tell from reading through the (quite pleasant!) documentation [0], Sylve does not (at least for now) support any sort of network storage for direct use as the VM backing store, though as it is FreeBSD underneath it's presumably doable to get something going from the command line. I'd thought they'd somehow managed to set something up so you could directly use another ZFS system via SSH as the primary backing store with management which would be pretty awesome. It still looks like a beautiful design, but since I'm pretty invested right now in separating out storage into its own hardware vs where compute happens it'd be hard to setup nodes as AIO for the near future at least here.

Still an awesome project to learn about and I hope it's successful.

----

0: https://sylve.io/docs/

[arch1e]

It's funny, I love how FreeBSD manages iSCSI even though I have only used it a few times, I put it in my to-do list but never really got around to writing a UI for it. Come next release (v0.3.0) I will definitely integrate it because as your put it's quite necessary to have that as a way to isolate storage from the main system.

[xoa]

Not sure you'll see this so late but just wanted to say I really appreciate the reply and learning about this project. I've been working to switch myself and various places away from perpetual ESXi licenses as it finally starts really getting old, and while I'm thankful Proxmox exists I've always loved FreeBSD (was kinda bummed when TrueNAS moved from it) and Proxmox can be irksome. Even at such an early stage Sylve already looks like it's clicking nicely. Excited to see next release and what comes in the future.

[TacticalCoder]

> They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.

A huge, totally obvious, advantage is that FreeBSD isn't using systemd. I'm now nearly systemd-free, if not for Proxmox. But my VMs are systemd free. And, by definition, my containers too (where basically the entire point is that there's a PID 1 for the service and that PID 1, in a container is not systemd).

So the last piece missing for me is getting rid of Proxmox because Proxmox is using systemd.

I was thinking about going straight to FreeBSD+bhyve (the hypervisor) but that felt a bit raw. FreeBSD+Sylve (using bhyve under the hood) seems to be, at long last, my way out of systemd.

I've got several servers at home with Proxmox but I never, on purpose, relied too much on Proxmox: I kept it to the bare minimum. I create VMs and use cloudinit and tried to have most of it automated and always made it with the idea of getting rid of Promox.

I've got nothing against Proxmox but fuck systemd. Just fuck that system.

[redserk]

Whether an appliance OS uses SystemD or not is as silly of a concern as “does the lead developer prefer cheddar or brie”

What about performance characteristics? Recoverability of workloads?

I’m interested in a FreeBSD base OS because it seems ZFS is better integrated and ZFS has a lot of incredibly useful tools that come with it. If Bhyve is at least nearly as performant as KVM, I’d be hard pressed not to give it a whirl.

[hedora]

It's not silly at all.

I've been repeatedly burned by systemd, both on machines I've administered and on appliances. In every situation, the right fix was either "switch distros" or "burn developer-months of work in a fire drill".

In fact, I just decided to go with FreeBSD instead of proxmox specifically because proxmox requires systemd. The last N systemd machines I've had the misfortune to touch were broken due to various systemd related issues. (For large values of N.)

I assume that means anything built on top of it is flaky + not stable enough for production use.

[pdntspa]

I have never really understood the systemd hate. It sure as hell beat the sorcery that was managing init.d scripts for everything.

I managed the distro upgrade on hundreds of remotely-managed nodes, porting our kiosk appliance from a pre-systemd debian to a post-systemd debian, and out of all the headaches we suffered systemd was not one of them, short of a few quirks we caught in our development process. It pretty much just worked and the services it provided made that upgrade so much easier.

Curious how you got burned, I hear a lot of complaining but haven't seen a lot of evidence

[fouc]

I don't understand the init.d script hate ;)

[redserk]

It absolutely is silly. I’ve been responsible for managing low-thousands of Linux servers with systemd and it’s standardized a lot of things that otherwise would’ve been a lot of bespoke scripts.

[arch1e]

Yeah, I’m kind of in the same camp. I never really had issues with systemd either. It mostly just works, even if it’s a bit heavy.

For me, moving to FreeBSD wasn’t about escaping systemd, it was more about the overall system design and how cohesive everything feels. That said, I’ve tried to keep Sylve neutral on that front. I don’t really position it as “systemd vs not”, just focus on what it actually does well.

It’s still early and not as feature complete as Proxmox yet, but I think it already stands on its own as a solid option.

[gerdesj]

“does the lead developer prefer cheddar or brie” Quite right but given I live in Somerset (UK) I can have both: Cheddar is in Somerset and where the eponymous cheese originated and quite a lot of brie is produced here too - it's not the French original effort but rather good.

I have quite a lot of customers that we have migrated from VMware to Proxmox. Some of them are rocking zfs instead of vmfs. Mostly these are Dell servers. Proxmox with zfs seems to be more aggressive about disc failure warnings, which I think is helpful.

Pick what OS works for you.

[wao0uuno]

What's wrong with systemd? It's easy to use and works well.

[Cyph0n]

Sometimes unification can be an advantage.

I run Proxmox at home, but now that I have been drinking the NixOS koolaid over the past 2 years, all of my homelab problems suddenly look like Nix-shaped nails.

[EnigmaCurry]

Same. Here's how I scratch the NixOS itch on Proxmox and/or libvirt[1]. One interface for both targets.

[1] https://github.com/EnigmaCurry/nixos-vm-template

[dizhn]

That feature list looks really good. It would actually be really nice to standardize the guest operating systems in such a way.

I actually have a few hosts that only run docker. I might be able to test with those.

[craftkiller]

Well it looks like we might soon be able to have the benefits of NixOS while also having bhyve (and presumably Sylve): https://github.com/nixos-bsd/nixbsd

[dizhn]

https://github.com/SaumonNet/proxmox-nixos

Looks like Nix will eat the world soon. :)

[Cyph0n]

Damn this is crazy!

[dizhn]

I have the same thing with proxmox especially after I realized how well it integrates with proxmox backup server. And I haven't even gotten into clustering yet. It really is a very solid product.

[Cyph0n]

Indeed, Proxmox VE is an amazing product.