[Tepix]

This could be an interesting setup for booting off a NAS like Synology or QNAP. I haven't really used iSCSI, it's intimidating how much prep this takes...

[rwmj]

iSCSI seems intentionally obscure. One of the improvements I made to NBD was invent a simple, standardized URI format so that you can specify servers easily, eg:

  nbdinfo nbd://server
  nbdcopy nbd://server:2001/ nbd+unix:///?socket=/tmp/localsock

https://github.com/NetworkBlockDevice/nbd/blob/master/doc/ur...

[codethief]

NBD looks pretty nice! I've been eyeing it from afar for a while.

How well does it work in environments with noticeable network latency?

[rwmj]

Requests and responses are pipelined so at least you're not serializing on round trips. However fundamentally if there is lots of latency, then you're going to be affected in some way. Usually we see problems where the OS accessing the remote drive times out which can sometimes be worked around by increasing timeouts, if you can work out how. (Latency is going to affect every block device protocol in about the same way)

You can actually see what happens quite easily if you've got an OS image handy. With a Fedora VM image:

  $ virt-builder fedora-42 --root-password=password:123456
  $ nbdkit file fedora-42.img --filter=delay rdelay=50ms \
           --run 'qemu-system-x86_64 -machine accel=kvm:tcg \
                     -cpu max -m 2048 \
                     -drive file="$uri",format=raw,if=virtio'

("$uri" expands to the NBD URI of the nbdkit server which qemu can parse natively)

Even that 1 second delay is painful since it turns out that booting is quite serialized. Edit: I turned down the latency to 50ms in the example which is a bit more realistic. Still painful.

[burner420042]

The 'target' moves slow so once you learn it, it all stays relevant forever.

... And it's very, very fun.

[Tepix]

Does it offer performance advantages over NFS root?

[a96]

I kind of expect the performance is worse, but one neat thing is that iscsi is a block device, so you could run e.g. disk crypto, volume management or whatever on it. Not to mention any FS. And you don't need to deal with NFS or RPC.

[burner420042]

They operate at different layers.

iscsi is a block device: you gain a 'disk drive' sitting on your network. A dedicated network for disk traffic and use it to host on-prem virtualization. It's called a SAN array.

[Tepix]

Sure, but eventually you will reach a layer with files, usually. Then you can run benchmarks and compare numbers.

[bobmcnamara]

Dunno about performance vs NFS, but I've stuffed an unaware OS onto ZVOL-over-iSCSI using a NIC with option ROM.