Hacker News new | ask | show | jobs
by glowingly 1537 days ago
Upstream kernel support is a bit sparse for the non-server boards.

When I was looking around, there was some stuff in the $500 to $2000 range:

Nvidia's usual Jetson/Tegra lineup

- sometimes has decent CPUs

- Good luck running anything other than Nvidia's slowly updated Kernel with blobs all over. Support drops real fast for older boards, leaving one stuck on old kernels (Jetson Nano and its upstream family was "EOL'd" in terms of kernel upgrades a while ago, even though Nvidia will still gladly take your money for a new Jetson Nano). - Has a lot of Nvidia stuff attached, so you do get a GPU and a PCIe slot.

NXP LayerScape

- lots of CPU cores, but not great ones

- Claims 2nd highest level of ARM Systemready, so it may work with mainline kernels?

- Has a lot of networking stuff attached, since this seems to be the successor to generations of PPC networking chips.

Apple

- IMO, you already know the pros and cons, and will have already decided to purchase a Mac M1 or not by now.

- Popular vendor with a tendency to not make too many different variants, so some are trying to mainline device support in the Linux kernel (Asahi linux).

Qualcomm's developer platform for WoA

- Somewhat limited in specs and afaik, no mainline kernel support of note.

There are others in this range, such as Amazon's Annapurna Labs, Marvell's various SoCs, Broadcomm, Ampere, etc. IMO, none of them really target consumers or workstations. Some (Marvell, Broadcomm) treat simple public datasheets and documentation as a sin - Marvell's takedowns of fmr XScale documentation in particular. Even Nvidia isn't quite this prudish with their documentation, though they do hold back a bit vs the big x86 giants. Annapurna is found all over the place (Qnap NAS & Mikrotik routers are two places I've been surprised to find them), so there may be reclaimed consumer hardware, but Amazon is similarly stingy with documentation.

Ultimately, I don't feel it is really the year of the ARM workstation quite yet.

1 comments

> Support drops real fast for older boards, leaving one stuck on old kernels (Jetson Nano and its upstream family was "EOL'd" in terms of kernel upgrades a while ago, even though Nvidia will still gladly take your money for a new Jetson Nano). - Has a lot of Nvidia stuff attached, so you do get a GPU and a PCIe slot.

Note that the Jetson Nano is supported pretty well by Fedora with a fully upstream kernel. This includes GPU acceleration through Nouveau, without reclocking catches.

And just before deprecating support for it entirely (won't get BSP releases beyond JetPack 4.x), they gave it u-boot on SPI with the UEFI module, which wasn't used at all before.

https://nullr0ute.com/2020/11/installing-fedora-on-the-nvidi...

The new BSP release that is released tomorrow in public preview is Xavier onwards only, and sets the baseline to Linux 5.10, with UEFI across the board.

The older BSP release that supports Tegra X1 onwards, including the Jetson Nano, will continue getting security updates for years to come. Just don't expect new features anymore on the NVIDIA binary UM driver stack.

> slowly updated Kernel with blobs all over

It's a heavily diverged kernel tree, but it's all GPLv2, including the full GPU kernel-mode driver (https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=sum...) for those. (there's no binary kernel modules present at all on the platform)

Firmware, like everyone else, and userspace is where you have the proprietary bits.

> It's a heavily diverged kernel tree, but it's all GPLv2, including the full GPU kernel-mode driver (https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=sum...) for those. (there's no binary kernel modules present at all on the platform)

> Firmware, like everyone else, and userspace is where you have the proprietary bits.

My experience with this, is moving away from the Nvidia kernel isn't practically feasible. Userspace may be a refuge, but even trying to upgrade away from the Nvidia/Ubuntu 18.04 userspace was always going to bring up general incompatibilities and small problems. The long awaited 5.10 update targets Ubuntu 20.04, just as 22.04 is progressing through beta. Backporting PCIe device drivers and modules was painful enough that I eventually gave up on my jetson adventures.

The nullr0ute URL even notes: > it’s not perfect yet and we’re actively working to fix and improve the support for the Jetson Nano [] https://nullr0ute.com/2020/11/installing-fedora-on-the-nvidi...

In my experience, perfect was too lofty of a goal for running a mainline kernel on a Jetson/Tegra. Similar to these users as well. [] https://lists.fedorahosted.org/archives/list/arm@lists.fedor...

[] https://forums.developer.nvidia.com/t/installing-centos-on-j...

In the end, I spent enough time trying to get the platform to work normally, that I realized I was wasting time when I could just get an x86 board for similar cost or get a Raspberry Pi CM4, and quit trying to grow the Jetson Nano out of its embedded system roots.

Nvidia needs to step it up, big time. Jetson Xavier users are finally going to get a beta for Cuda 11, when x86 and SBSA arm users had Nvidia's official Cuda 11 for almost 2 years now.

> My experience with this, is moving away from the Nvidia kernel isn't practically feasible

Oh yup... for just 4.14 (from the NV 4.9 kernel), took so much work to keep up...

I eventually had a project to decouple nvgpu so that it could be used with a regular mainline kernel as a DKMS module... but I didn't get around to it. The situation on that front will become better in the future, but it's taking way too long.

> Userspace may be a refuge, but even trying to upgrade away from the Nvidia/Ubuntu 18.04 userspace was always going to bring up general incompatibilities and small problems.

CUDA 10.2 doesn't support Ubuntu 20.04 so hello problems on that front. Enough said...

> Nvidia needs to step it up, big time. Jetson Xavier users are finally going to get a beta for Cuda 11, when x86 and SBSA arm users had Nvidia's official Cuda 11 for almost 2 years now.

Yes... the thing is that Jetson has been put exclusively on an LTS lifecycle and that posed a bunch of problems. We have promises that it won't happen again in the future, with CUDA being decoupled from L4T shipping sometime towards the end of the year.

And the beta shipping tomorrow is for CUDA 11.4, not the latest 11.6.

Also as a side note if you're on Xavier and trying to OTA update to the BSP released tomorrow, just don't, that flow is just not supported. Full reinstall required between major BSP releases.