[timost]

A few elements to inform the decision:

- Both docker and podman support rootless containers.

- Rootless podman setup is easier to achieve from experience and it integrates well enough with systemd.

- Docker requires a daemon to run at all times whereas podman doesn't.

- A lot of interesting things are going on with podman ("native" gitlab-runner executor in the works, wsl2 support, among other things)

[eberkund]

Doesn't Docker already support WSL2? Or is there something different about Podman's WSL2 support which makes it better/different than/from Docker's?

[lmeyerov]

Yep, docker in wsl2 is great, including GPU support

wsl2 mode is super impressive -- none of the overhead of docker on os x

[girvo]

None? How does it manage that, it’s still running in a (Hyper-V) VM?

Are there any good docs/blogs that go into detail on how they’ve managed to avoid that overhead? Would be awesome to learn about

[lmeyerov]

For the comparison to os x docker, the biggest slowdown for os x docker is the file system mapping. It basically breaks any reasonable performance chart: look up posts getting into alternatives like mutagen. The slowdown makes using an IO-heavy dev flow like live editing via `npm run watch` relatively unusable: 30X+ slowdowns are normal. There might be high CPU overheads too (I recall we had to tweak seccomp), but when basic IO is so slow, the rest doesn't matter.

In contrast, yes, wsl2 has 5% CPU hits (hyperv, ...),but a sane FS mapping, so the total overhead is imperceptible for a windows dev box.

I was pleasantly surprised to see wsl2 Just Work. Our only issue preventing wsl2 from being the official team rec over native Linux has been wsl2's lack of opencl, and that's just specific to our use of GPUs. As someone whose preferred dev box has been osx for ~20 years, even when at MS, I was biased against Windows for most dev... but no longer.

[girvo]

Oh right yeah — to be honest because I was so used to file system mapping being so slow (and even on my W10 box it’s not perfect for deep file system trees like node_modules), I stopped using it.

VSCodes “Create Remote Dev Container from Repository” functionality has made it even easier. My dev containers have no overlap with the host filesystem so macOS and Windows are equally performant for my use cases :)

[lmeyerov]

Yes, wsl2 afaict works like that and a bit improved. Instead of dual booting or a restricted dev env, I click my Linux terminal icon and jump into a full & fast Ubuntu, including UIs when helpful. Sharing with Windows is possible through a shared mount of C:, but I rarely encounter it, basically only for a web downloads folder or working around certs. Same thing for VS Code, have a Windows mode and a wsl2 mode, and it opens the right one for you.

Almost all our dev is generic, so that means Windows for Office/web/... and full Linux for dev (except no real OpenCL).

[politelemon]

Just curiosity, what are you using the GPUs for?

[lmeyerov]

interactive graph ai + graph viz, such as for understanding it/sec/fraud/customer log activity

[speedgoose]

HyperV is lightweight and from what I understood, windows runs on top of hyperV too when you enable WSL2. You may lose a few % of performance in video games.

[shaklee3]

any idea if podman will support GPU? I couldn't get it to work.

[lmeyerov]

In theory yes, and we got tiny programs running this way, but could not afford to keep subsidizing megacorp politics.

Nvidia punts to IBM RHEL8 docs for GPU podman, which is unusual and risky to see. We officially recommend against it for HA environments due to this kind of lack and overall low relative confidence. I think k8s envs may be moving to something here, so maybe in a year or two? I'd be curious of folks doing stock rhel8 podman with tensorflow/torch on nvidia, which should be as vanilla as you can get for enterprise ai. We generally see more interesting GPU envs here (ex: DGX with advanced networking hw/sw), but we don't have confidence for the simple case, which is the starting point..

[timost]

Docker supports wsl2 but I have constant performance issues with it (memory leak, high CPU usage).

[formerly_proven]

podman indeed has many interesting features, but those features tend to have interesting bugs and limitations as well.

[timost]

What got me at first are the limitations around rootless containers, which I wasn't accustomed to. Rootfull docker makes many things (networking, volumes etc.) Fairly easy.

I've found this troubleshooting page quite helpful: https://github.com/containers/podman/blob/main/troubleshooti...

[arjvik]

Any specific limitations? I want to understand how much of a deal breaker they are.

[op00to]

All software has bugs. All software has limitations.

[rantoie]

Where can I find more information about this 'native' gitlab-runner executor?

[timost]

I'm following this issue:

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27119

[rantoie]

thanks!

[timost]

Npb. It's also listed on the upcomming release page apparently

https://about.gitlab.com/upcoming-releases/