[jandrese]

Sadly PXE boot of a modern Linux OS is easier said than done. Gone are the days of just handing out the kernel over tftp and providing a NFS root. SystemD gets super cranky and you can't even boot the thing without setting some undocumented flags.

[hpcjoe]

A little tooting of my own horn here, but nyble[1] allows you to build a ramdisk bootable image and kernel, that you can serve trivially with any http server and iPXE. A related project, tiburon[2] allows you to make this JSON db controlled. Still working on documenting them in greater depth, but you can see an example of what nyble can do[3].

[1] https://github.com/joelandman/nyble

[2] https://github.com/joelandman/tiburon

[3] https://scalability.org/2019/05/nyble-ftw-installing-my-ramb...

[marmaduke]

What? I just did this today with Ubuntu 18, using dnsmasq and a nfsroot based on a virt-builder image. It runs fine..

[ryanmjacobs]

I highly recommend using pixiecore as an iPXE server -- your remote machines only need to have PXE enabled. Previously, it was a pain in the ass to install the TFTP server and get a DHCP server capable of not binding to every request (and only PXE requests like the standard allows). pixiecore does everything for you in like 10 seconds: https://github.com/danderson/netboot/tree/master/pixiecore. We're using it on-premise to spin up a server rack into stateless Kubernetes nodes. None of the blades have a hard-drive/ssd :)

[naner]

PXE booting Linux live media is fairly straight-forward. Grub/syslinux syntax translates pretty easily to iPxe (which is simply a different type of bootloader). You just provide a kernel,a ramdisk, and whatever kernel arguments you need. If you're trying to roll your own ramdisk environment from scratch you may run into trouble but in general network booting Linux isn't any trickier than making a bootable iso.

[gmueckl]

How did the systemd cabal manage to mess that up? I ditched that thing a while ago for good reasons, so I don't keep track anymore.

[jandrese]

I'm not entirely sure of everything, but problems with UUIDs (which we expected), and some dbus signals not being generated when run over a NFS mounted root causing the boot to hang. We hacked some timeouts to get around the problem but never figured out exactly where the signals were supposed to be generated from.

A hint if you're doing this on Linux. We PXEBoot an iPXE loader to boot the machines. Doesn't work properly on UEFI unfortunately, gotta use BIOS boot.

If it helps, I have notes on how to set that up:

Go to http://rom-o-matic.net and choose gPXE git. Click on the "Customize" button to expand all of the options.

Choose: 1. PXE bootstrap loader image [Unload PXE stack] (.pxe)

2. all-drivers

3. PCI VENDOR CODE: [blank] PCI DEVICE CODE: [blank]

X CONSOLE_PCBIOS

_ CONSOLE_SERIAL

BANNER_TIMEOUT [20]

_ NET_PROTO_IPV6

(Serial Port Options are irrelevant)

X DOWNLOAD_PROTO_TFTP

X DOWNLOAD_PROTO_HTTP

_ DOWNLOAD_PROTO_HTTPS

_ DOWNLOAD_PROTO_FTP

_ SANBOOT_PROTO_ISCSI

_ SANBOOT_PROTO_AOE

X DNS_RESOLVER

X IMAGE_ELF

X IMAGE_NBI

X IMAGE_MULTIBOOT

X IMAGE_PXE

X IMAGE_SCRIPT

X IMAGE_BZIMAGE

X IMAGE_COMBOOT

X AUTOBOOT_CMD

X NVO_CMD

X CONFIG_CMD

X IFMGMT_CMD

X IWMGMT_CMD

X ROUTE_CMD

X IMAGE_CMD

X DHCP_CMD

_ SANBOOT_CMD

X LOGIN_CMD

_ TIME_CMD

_ DIGEST_CMD

X PXE_CMD

_ IPV6_CMD

_ CRYPTO_80211_WEP

_ CRYPTO_80211_WPA

_ CRYPTO_80211_WPA2

Embedded Script:

-----------------------------------------------------------------------------

#!gpxe

dhcp any

initrd http://<your_server_here>/initrd.img

kernel http://<your_server_here>/pxelinux.0

imgargs pxelinux.0 root=/dev/nfs rw boot=nfs nfsroot=<your_nfs_server_here>:/netroot root ip=dhcp nfsrootdebug

boot pxelinux.0

-----------------------------------------------------------------------------