[derefr]

> IPMI makes it easy to setup over LAN and I don’t have to ever plug in a monitor or keyboard.

Hardware BMCs have their place (e.g. low-overhead compute-cluster nodes, where free cores = profit.)

But, for most workloads—and especially consumer workloads—there’s no reason that the concept of a “Baseboard Management Controller” needs to be instantiated as hardware; you can just as well set the system up with a hypervisor OS (e.g. a minimal Linux KVM install; or an appliance-OS designed for this, like VMWare’s ESXi), set your regular workload up as one VM guest (and pass through to it all the nice hardware you have, like GPUs), and then set up another “control plane” guest VM that exposes IPMI management of your regular guest and of the hypervisor itself. As they say, “there’s no problem that can’t be solved with another layer of indirection.” ;)

(I should note, this is exactly the setup you get by default if you install ESXi [hypervisor] + a free home license of vSphere Server Center [BMC-equivalent appliance] onto a box. I was happily using this exact setup for quite a while, though I eventually moved to Linux+KVM+Xen just because I wanted the host to be able to create guest volumes from a thin-provisioned storage pool and then serve them out to the guests over iSCSI, as if I had a teeny-tiny SAN.)

Of course, this has only become a viable approach for IoT integrators very recently, which is why we don’t see any IoT appliances (e.g. NASes) coming set up this way from the manufacturer just yet. Until recently, your choices for building IoT devices were microcontrollers at the low end; old ARM cores in the middle; and Intel’s most “power efficient”, feature-stripped cores on the high end. None of these were particularly suited to hosting virtualization. But Ryzen is! While it may only be affordable to home-builders today, I expect to see AMD chasing Intel up on its “power-efficient embedded profile” market segment quite soon, with Ryzen-based, highly-cored, virtualization-capable equivalents to the Intel Atom line being sold for cheap enough to get system integrators excited.

[wil421]

FreeNAS does not recommend running in a VM and I’ve heard about problems with iSCSI :-). I could easily pick up used Dell servers dual core E5 Xeons with 128gb of ecc RAM and whatever SATA/SAS controllers I want off Craigslist. ESXi costs money and a yearly cost at that but I have played around with the trial version.

But! The FreeNAS community is a bunch of grumpy sys admins. I’m considering going down the Linux and ZFS route. I’d be able to do more with VMs (I feel more comfortable in Linux vs FreeBSD). I’m building some IoT Pi’s to collect data and have it a Linux box would be nice.

The UniFi USG handles DynaDNS and my VPN.

[technofiend]

Raspberry Pis haven't quite got there yet but I'm hoping the next iteration will have an NVME or SATA implementation. Although to be honest it doesn't have to be the Pi. Any small board that'll run Linux, has at least gigabit ethernet and a fast path to disk will do. At that point it'll be possible to make a ceph cluster with one Pi per disk.

[vetinari]

For some time, I've been thinking of making a ceph cluster out of ODROID-HC1 or 2 (https://www.hardkernel.com/shop/odroid-hc2-home-cloud-two).

[technofiend]

A few years go now Western Digital demonstrated an onboard controller with two 1 GB NICs and a mini linux distribution with a single Ceph OSD installed. Unfortunately it never made it out of the lab. I would gladly pay a $50 premium per device for spinning rust to have that onboard. Perhaps the issue is with NVME-connected devices that could be a much costlier device to build? Or maybe there's no standard for housing network-connected storage devices in a rack?

[_jal]

You can do that, but they're not really replacements for one another, and there are lots of things that can pull one way or another for that use case.

- You generally don't want to run storage servers virtualized.

- Tooling matters. There are multiple reasons I generally do things the same way at home as I do at work (within reason).

- Probably a niche concern, but I have some hardware that is only configurable during early boot.

- Virtualization costs performance. Not a huge issue at home, granted, and you have to quantify it for your specific workload. (It is usually going to be IO.) But it certainly can matter with home workloads; home theater video processing is probably the most common.

I use both for what they're good at. IMPI is for managing hardware. Virtualization is for not needing more of it.

[nwmcsween]

Why not a network kvm instance? There is lots of hardware out there that can do this and it doesn't limit you to the IPMI integrated hardware