[munro]

Does this mean I can run containers natively on Mac? And I don't need a VirtualBox VM running on my Mac to launch containers? This would be huge for me, and is always a big in my mind why I would consider switching back to Linux.

Edit: Docker on Mac has never felt as snappy as on Linux, because of the VM, though I have no hard numbers. Networking is a PITA, but it's not hard to figure out. The other main thing I hate is I have to give up a bunch of RAM to the VM that my containers may or may not use, instead of sharing with the host like on Linux.

[outworlder]

> Docker on Mac has never felt as snappy as on Linux

It's extremely slow compared to Linux and I'm pointing my fingers at the virtualization layer without any hard evidence because it's the most likely suspect.

With all this focus on sandboxing apps of late, I'm wondering how far the OSX kernel is from having a feature set that resembles cgroups and network namespaces.

[ploxiln]

I used docker inside a vagrant+virtualbox VM running ubuntu, on macOS, for a few years. It's more reliable, and more debuggable, than docker for mac. It's some easy-auto-transparent storage and networking layers that make docker-for-mac so flaky.

[GGfpc]

So you have your setup documented anywhere?

[my123]

They have containers already in a sense, whatever the iOS Simulator.app uses is not VMs, and it's a container in more ways than just one.

[astrange]

It's a launchd namespace. It's not meant to be secure in the same way a container would be, but it could be used for something.

[BillinghamJ]

That is not exactly designed to deliver any kind of isolation though - it just runs normal processes

[jontro]

Under the hood there is a linux vm running.

[diveanon]

Too bad, if they had it running natively I would actively consider switching back to mac.

Glad to see they are making progress on the M1 port though, my team will be excited.

[stephenr]

... if docker ran “natively” it’d mean using kernel hooks provided by xnu, which means you’d be able to run another instance of macOS in a container.

How is it so many people use docker but have zero fucking clue about how it works?

[haswell]

Docker obscures the implementation details. Part of this is by design: don't worry about the hard stuff, because Docker does it for you.

It's completely unsurprising that many folks haven't spent the time to dive into the internals. In many cases, because they don't need to.

Such is the nature of abstraction and higher level frameworks.

[outworlder]

We aren't even close to a point where someone can just pretend a container is a magical box that just works.

All abstraction leak. This is a relatively recent, quite leaky one.

[haswell]

> We aren't even close to a point where someone can just pretend a container is a magical box that just works.

I challenge this assertion. While it is of course true that there are situations that require a deeper knowledge of Docker, this is also not universally true.

Many projects these days have a Getting Started doc that has some options like:

- Build from source

- Install this package

- Use Docker

I often choose the Docker option because I know I'll (most likely) get a working version of the project in 5 minutes with minimal effort. I might not even fully understand the project yet or its architecture (much less that of Docker), but I can get it up and running with `docker pull` and `docker run`.

In many cases, I'll never need to know anything more.

I've personally spent more time using docker to build my own stuff, so I've had to learn more. But for many folks, it absolutely is a magical box that just works, and that's perfectly ok.

I agree that all abstractions tend to eventually leak. But depending on why you're using Docker, you may never have a reason to encounter that leakage.

[stephenr]

Just because CorporationX says “don’t worry about it we got you bruh” doesn’t mean technologists - people who actively work with technology and write software - should be excused for just throwing up their hands and saying “it’s just ducking magic I don’t know how it works”.

I’m not talking about understanding it to the level of being able to contribute a patch to the project.

I’m talking about understanding that containers are inherently tied to the kernel, and thus are limited to running software written for the same kernel as the host running the container.

It isn’t rocket science. I literally explained it in one sentence, and I’ve never used docker in my life.

This is along the same level of knowledge as “no, you can just take an iPhone app and run it on android” or “no you can’t just take a SQLServer query and run it on anything that vaguely knows “SQL”.

[haswell]

> Just because CorporationX says “don’t worry about it we got you bruh” doesn’t mean technologists - people who actively work with technology and write software - should be excused for just throwing up their hands and saying “it’s just ducking magic I don’t know how it works”.

Entire industries are built on the premise that "don't worry about it, we got you". I'm not saying that it's appropriate to be completely blind/unaware of what you're using, but there's a line somewhere that's surprisingly difficult to draw in 2020.

I don't think anyone would argue that learning more is a bad thing. But the more salient point is that for many, it's just not necessary.

If you're doing work that requires a deeper knowledge of the thing, then of course you should learn it. If you're not doing work that requires this knowledge, it'd be a waste of time, the most precious commodity available to us.

Others have made the comparison to learning assembly. Useful? sure. Necessary in 2020? Usually not.

> It isn’t rocket science. I literally explained it in one sentence, and I’ve never used docker in my life.

This is what you said:

> ... if docker ran “natively” it’d mean using kernel hooks provided by xnu, which means you’d be able to run another instance of macOS in a container.

Not only does this tell the reader nothing practical about how Docker actually works, it doesn't even address the parent comment in a useful/informative way.

You followed this up with a statement that is a borderline personal attack on the parent comment.

I mean this as constructively as possible, but you need to work on your delivery, and ask yourself what you're trying to accomplish with these comments. So far, they've been unhelpful and borderline abusive.

[geofft]

> “no you can’t just take a SQLServer query and run it on anything that vaguely knows “SQL”.

Not with that attitude you can't. https://aws.amazon.com/blogs/opensource/want-more-postgresql...

The world depends on people who don't take "no you can't" for an answer.

[WrtCdEvrydy]

Because we use abstraction as a way of lowering barrier to entry to reduce the requirements for people to be proficient in the career and be able to reduce overall costs downwards?

[NikolaeVarius]

Lowering the entrance requirements doesn't mean you should STAY at the entrance.

[bamboleo]

You keep going, I’ll take another door.

I know enough Docker to create and launch a dev environment, but I don’t need anything more than that since I don’t do deployments.

Likewise you don’t need to learn about browser rendering since you barely interact with it.

[Ar-Curunir]

Do you know the details of the x86 architecture? Why so toxic?

[geofft]

> which means you’d be able to run another instance of macOS in a container

This is not true, for multiple reasons. Strictly speaking it only means you'd be able to run another instance of Darwin in a container. And, as you surely know because your tone of voice implies you bear immense knowledge, a Docker-style container is not a full OS: it doesn't run an init or normal system daemons, so it wouldn't even be a full instance of Darwin, so it wouldn't have to support functionality only needed by launchd or system daemons (e.g. WindowServer). It would just need to let you run a stanalone program in a chroot + separate network, PID, and IPC namespace + apply resource controls.

Furthermore, since most people are using Docker for developing software that's going to run on Linux, there would be no real need to virtualize the parts of XNU that aren't also provided on Linux - notably all the Mach stuff. You'd just need to provide a BSD-style syscall API to programs in a container.

[jsjohnst]

> it doesn't run an init or normal system daemons

There’s no technical limitation stopping you from running init or system daemons inside a container, it’s just an anti-pattern and missing the point of a container in most cases.

[Groxx]

Because it's not necessary to know for day-to-day work. Which is what most software aspires to achieve.

[pitterpatter]

For the same reason that the general dev doesn't have in-depth knowledge about every OS they use: abstractions.

[dabeeeenster]

Because you don't need to?

[diveanon]

It's called abstraction, and it is what enables you to live in modern society without knowing the technical details of every facet of your life.

Furthermore, not being able to hook into xnu to virtualize macos is a business decision, not a technical limitation.

[paxys]

Docker can't run natively since it uses linux kernel features not available on a mac. Apple would need to do a lot of work first to make it happen.

[perbu]

If Docker for mac was running natively it would be more or less useless. Nobody deploys production code on MacOS hosts anymore.

That it is all so smoothly powered by Linux is what makes it a great product.

[diveanon]

I develop exclusively on Linux, but it would be nice if my teammates who don't didn't melt their laptops doing local dev on mac.

If apple gave a shit about non apple developers they would provide the kernel hooks to help make it possible.

Also, nobody deploys on mac hosts because virtualizing macos in a cloud environment is against tos, so options are expensive and limited.

[pitterpatter]

I mean, they do provide the hooks via Hypervisor.framework. I'm not 100% familiar, but if it's anything like KVM then running a linux VM like that shouldn't have that much overhead.

[ehutch79]

I deploy on linux, but dev on macOS without a vm or docker or anything. If you're not doing anything OS dependant, which most web apps don't, you can run everything natively.

[diveanon]

Isolation is another important reason people use docker.

I don't want my developers to have to manage a complex dev stack and it's dependencies.

I would argue that this is the primary use case for local docker dev these days.

[perbu]

Me and I just about everyone I know that has a mac develops _for_ Linux. What is nice is that I can push, pull and run Linux images on my mac.

If the containers where native Macos docker images it would be about as useful as native Docker on Windows. Which I'm sure is great for the few ppl that need it - but pretty useless for most ppl.

But I sure wouldn't mind if was a bit snappier. But it is plenty fast enough for my needs atm.

[diveanon]

I believe they would just need to virtualize what isn't already provided by xnu-darwin.

This may already be the case, but the memory allocation for docker vm on mac really handicaps things.

[gigatexal]

You wanted Apple to add container support to the OSX kernel? Hah. I wonder if the virtualization API that Apple is pushing performs better than Hyperkit.

[stephenr]

> HyperKit currently only supports macOS using the Hypervisor.framework.

Do people really not know how shit works these days?

[monkpit]

It’s great that you know about how Docker works. How about being a little more informative in your comments here?

[macintux]

Your hostility towards everything and everyone doesn't engender productive discussions.

[recursive]

If you've got something to say, just spit it out.

[munro]

People griefing about to you that you have no idea what you're talking about, smh, don't listen to them. All I have to say to them is: look at the Wine project running Windows software natively on Linux. Don't underestimate nerds who have a vision in mind. And look at Kubernetes deprecating Docker. At the highest level of application development, all these details don't matter. I'm using all GNU command line tools compiled for my Mac, I'm sure we could figure something out to increase containerization efficiency on Mac. ¯\_(ツ)_/¯

[munro]

Dang, wishful thinking that they were going native on M1. ;_;

[jcollins]

Docker Desktop already ran on Macs. This is specifically for the new Apple Silicon support (M1). It's not native, technically, but it feels native the way Docker Desktop works. Basically they manage the VM for you, so you don't have too.

[diveanon]

It doesnt feel native at all, performance and networking are very suboptimal.

Haven't tested it on M1 yet, but I doubt the networking challenges will disappear.

[jcollins]

There is overhead for sure but if you use x86 containers on an Intel Mac or arm64 containers on an Apple Silicon mac, it's pretty performant.

[diveanon]

Can you share any details on M1 perf and resource consumption? Docker vm on mac is noticeably slower and consumes a lot of resources in my experience.

[jcollins]

The only thing I can say with confidence is that if you're using ARM containers it is really fast, probably thanks to the Apple Silicon.

I imagine memory consumption is on par with running on Intel. I don't think Docker Desktop can really change that.

[diveanon]

Honestly memory is the primary constraint for most of my mac devs running our docker envs.

[dalacv]

Can this run a container of windows on Mac M1?

[jsjohnst]

Can you run a container of Windows on an x86 machine? The answer is no, and for the same reason it won’t work on ARM. A “container” is not a virtual machine, you can only run the same Linux executables you would on a normal Linux system.

That said, as another person commented, you can run Windows for ARM in a VM on an Apple M1.

[thefourthchime]

You can run Windows For ARM on M1 Macs right now, I doubt you can get docker to do this without lots of manual effort.

[gigatexal]

Got a link to proof of this?

[my123]

https://www.mail-archive.com/[e-mail]/msg766282... for the Qemu patches

and https://www.microsoft.com/en-us/software-download/windowsins... for downloading the Windows build

[dalacv]

Question, is there persistence with that? Or are changes lost once it is closed?

[my123]

No nested virtualisation present currently, as such no virtualization support provided to VMs, so on Windows on an M1 only WSL1 works. Docker Linux containers on Windows require WSL2 instead.

Docker Windows containers aren't available on arm64 Windows yet, but stay tuned...

[shimylining]

Im confused by this comment? I am running Docker Desktop on Mac and have multiple containers running?

[diveanon]

It is running in a vm

[paxys]

Docker has never needed a VirtualBox VM to launch containers on your Mac. It does its own virtualization internally, and will continue to do so.

[Wowfunhappy]

It actually did at one point; before Docket Desktop there was “Docker Toolbox”, which required separate virtualization software. The installer came with Virtualbox by default, but there were options to use Parallels and VMWare as well. This is probably what GP is thinking of.

I'm actually using a version of this setup today in order to run Docker on OS X 10.9.