[userbinator]

Even in Windows 95, which didn't support USB at the time.

That's because DOS-based Windows could use the BIOS for disk access, and BIOS presented USB drives as hard drives. I believe you can even do the same with an NVMe SSD that has a suitable boot ROM.

[fuzzfactor]

Yes, good to emphasize that UEFI or genuine BIOS motherboards will access the USB drives on powerup, then any OS that can boot from that type partition layout can go forward from there. DOS, W9x, NT5 need CSM enabled to boot on a UEFI MB, W7 loves it as well.

W98 would install and run from USB too, as long as USB device drivers did not get installed. That way once booted if you plugged something into a USB socket on the MB, it was "unknown" and remained inaccessible. But if you booted when the second USB device was plugged in beforehand, W9x (or DOS) assigned an alphabetic drive letter and you could access the files.

Sometimes I still use a small FAT32 partition with simple DOS on a Syslinux'ed volume to boot distros from the NT5 bootloader. That way you can edit the Linux multiboot menu in Windows, or even DOS which sure boots a lot faster today.

Now they have USB enclosures for M.2 drives, usually not both NVMe & SATA flexibility though.