Hacker News new | ask | show | jobs
by mulcahey 2006 days ago
Perhaps a dumb question but I'm curious why people don't use a VPS or a cloud linux machine more for Docker/K8s development instead of running Docker locally on a Mac.

In my experience Docker Desktop has been such a resource hog, and Apple's hypervisor implementation pretty poor. I much prefer to have all that heavy lifting isolated away from my development machine to keep it responsive and cool.

10 comments

If you have multiple developers working on the same product, you want each developer to have their own environment. In my experience, for cost, convenience, and productivity its made most sense to do this with local Docker. It also eliminates a lot of connectivity issues with running on a remote machine.

> In my experience Docker Desktop has been such a resource hog

Not sure if this is related to the specific project or what, but I haven't had tons of trouble with docker being a big hog.

That said, they are significantly changing the way Docker on M-Series CPUs works, working directly with Apple's Virtualization Framework which I believe improves performance.

Here is an example thread of many users experiencing resource utilization issues with various root causes:

https://github.com/docker/for-mac/issues/3499

Some root causes have been addressed. If they are significantly changing the architecture with the Apple Silicon rollout, I'm very much looking forward to see if that improves things!

Docker for Mac is a really bad resource hog for devs at my company. It's a common refrain that comes up again and again.

We build a SaaS web app with a MERN stack deployed to AWS. Local development usually involves running about 12 containers.

Personally, I got fed up and installed Ubuntu 20.04 in Parallels and have been running the Docker containers there for several months. What a night and day difference... the entire VM sits at about 13% CPU when idling, whereas Docker for Mac would whine my fans at around 150% CPU continuously when idling, and spike up to 300+% when clicking around the app.

I wasn't trying to argue the point about Docker being a resource hog! I've always run Docker on computers with lots of RAM or with fairly simple images (or both sometimes) so it's likely I've just dodged that bullet.
Back in the old days, each of those developers would just use telnet and X Window sessions on the same UNIX box, which the cloud is nothing else than the same stack in new clothing.
The main resource issue is because Docker runs in a VM, it can't share memory with the OS. So it's using the configured amount of memory (2GB by default) whether you're running busybox or an entire Kubernetes cluster.
I think the Virtualization Framework mitigates that to some extent. The way Docker was required to run on x86 it needed a Linux VM which I believe the Docker images ran atop. Under the new model, I think they run on bare metal and share memory.

Regardless, for me the cost of adding enough RAM to support Docker is less than the headache and expense of spinning up Docker images in the cloud.

M1 Docker still needs a Linux VM, Virtualization Framework is just a different way of doing that. I'm not aware of any VM that can share memory with the host, but maybe Apple has some black magic up their sleeve.
Yeah, fundamentally Docker is Linux on Linux. If you are running Linux on MacOS, it's not Docker, it's a VM. So Docker installs are Linux Docker Image -> Linux VM -> MacOS Virtualization Framework.

For that reason alone, I suspect Linux will always be the best host system for Docker images.

On Windows Docker uses Windows containers for Windows images and WSL for Linux images, so whatever is the best host depends on the images.
> I suspect Linux will always be the best host system for Docker images.

It seems that for more than 1-2 containers, running Docker inside a Linux VM on macOS is better resource-wise.

Seems like Apple's Virtualization Framework supports some sort of memory ballooning mechanism (https://developer.apple.com/documentation/virtualization/mem...), which could hopefully enables in the near future dynamic memory support like Docker Desktop on WSL 2 is able to do (if it's not already the case, I haven't tried the M1 preview).
If I'm writing code on a shitty laptop keyboard instead of a proper mechanical one, it's because I am, or intend to often be, outside an office, quite possibly travelling. This also usually means that I will not have a reliable internet connection.
It can get surprisingly expensive when left on 24/7. If someone can come up with a remote (dare I say, serverless?) development environment that could seamlessly suspend/resume or share resources, then it would be a lot more compelling to me.
AWS Cloud9 has a setting to shut down inactive instances after X number of hours. Then you aren't charged for the compute time, only storage.
That's one of the premises behind the Codespaces offering: https://github.com/features/codespaces
I'm pretty excited about this and see if being the norm for most web dev in a few years.
Network volitilty and debugging were the reasons I moved back to a mostly local dev setup.

I also run linux though so resources aren't much of an issue for me. I'm actively considering setting up remote environments for one of my teams that is all mac users working in a containerful dev environment.

Without kubernetes, Docker Desktop is fine. Been running it for years on my current and previous mac book pro (the 2012 model). Docker performance has never really been an issue for me. The bundled kubernetes seems to burn a lot of CPU though; so I keep that disabled.

I work with other people in teams and I don't get to tune every project I deal with to be just right for my tastes and hardware. So being able to run their stuff as is with emulation is preferable to me having to customize everything before I get to run it.

Clarifying a bit - I can't start up the Docker Desktop Kubernetes - If I take a look at the image downloaded (docker/desktop-kubernetes) it only shows tags for amd64.

When I try to spin up kubernetes, the UI just shows 'starting...' while the logs show the image won't work with this architecture.

Anyone else see this? Any workaround?

It's been taking a while, and the kubernetes is still 'starting...' - Did you have the same issue?
Probably because it’s not very convenient or easy to setup. I have a 2013 i7 MacBook Pro and I don’t have that much trouble running Docker with 16GB of RAM.
I’m building a startup that aims to help with this - but as others have mentioned - who wants to pay for two computers?

One solution is re-purposing an old PC into a home-server - once that’s setup, tools like Skaffold start to shine - local network speed, a cool laptop, containerized development, and a platform for home-hosting!

Anyways I’m very excited about this future - much more than paying AWS for another machine!

cost? if im dropping $3k on a machine, I don't really wanna pay another $2k in cloud fees to do the dev work.

you might say, "well get a cheaper computer!" but I find that cheaper computers have worse screens, battery life, and overall quality.

I don't want to be _too_ pedantic but the M1 macbooks are significantly cheaper than their Intel counterparts. The 13 inch one is around $1300 USD.

But I do agree with you. Cheaper laptops means a worse experience with almost everything.

> I don't want to be _too_ pedantic but the M1 macbooks are significantly cheaper than their Intel counterparts.

The M1 MBP is about 15% cheaper, which I guess might be significant to some, but isn’t a huge selling point to me like you’re making it out to be. GP was talking about a 15/16” MBP at $3k, not the 13”.

Developers for the most part have or can look forward enough to enough income to make a tool like a mac relatively inexpensive. My biggest problem with buying them has been how absurdly bad the hardware has been for years other than the trackpad. The screensize/weight ratio, the screensize/body size ratio, the keyboards, and the performance were all well behind the competition. MacOS only running natively on bad hardware was a great reason not to use it, the fact they charged a $500+ premium on their software was really just another nail in the coffin. You had to need or fucking love MacOS to justify buying a mac.

How things change in a year or two. Pretty much only the weight/size relative to the screen size remains substandard and I expect them to improve on those fronts.

> The screensize/weight ratio, the screensize/body size ratio

Based on HN comments, one of the most popular non-Apple laptops seems to be the Dell XPS.

XPS: via https://downloads.dell.com/manuals/all-products/esuprt_lapto...

- 15.61”

- 4.14lbs

- overall dimensions 14.06 x 9.27 x 0.66in or about 86in^3

Apple 16” MBP: via Apple.com

- 16” screen

- 4.3lbs

- overall dimensions 14.09in x 9.68in x 0.64in or about 87.3in^3

—-

So weight/screensize ratio:

Dell = 0.27lb/in vs Apple = 0.27lb/in

Screensize/bodysize ratio (higher is better):

Dell = 92.7% coverage vs Apple = 93.6% coverage

So just using your first two arbitrary criteria, seems Apple’s 16” laptop is ahead of or tied with, but not “well behind”, this competitor.

I’m sure you can pull out some edge case laptop that beats it, but as the XPS is arguably the most popular non-Apple laptop among devs, I feel it’s fair to say the Apple laptop is definitively not “well behind the competition”, at least using your criteria.

For a laptop that’s <3oz heavier and only 1.3in^3 larger dimensions, you get a screen almost a half inch bigger, almost 20% larger battery, the worlds best trackpad, 8 core i9 vs a 6 core i7 processor, option to get 64gb ram rather than being limited to only 32gb, etc etc.

The 16" slipped my mind, that varient to me was the first "good" laptop apple released in years but they were pushing a dated 15" model not so long ago. This laptop had quite a bit of overheating issues to the point you can't even drive all the monitors it says it supports on the spec sheet off the iGPU anywhere close to comfortably.

The 13" models which I'm more interested in are more dated and more behind in terms of weight although the performance makes up for that. What truly stunned me a few years back was comparing a 13" MBA to a T480s and seeing Macs just get tied or crushed in pretty much any area that wasn't related to speaker quality or the trackpad. This laptop had an extra inch of screen space and an extra 2 cores and better input overall and higher memory support and way more ports and about the same battery life and the same weight and all this for substantially less money especially if you bought memory aftermarket. It was nothing short of a humiliation - and this wasn't even Lenovo's top product - Apple wasn't even competing with second rate products. I seriously wondered how Apple had fallen so far from the heyday of the MBPr and if it was just going to let the MacBooks decay into irrelevance.

I also would compare Apple to the X1 line, not the cheaper XPS line with a 32gb memory limit just because it's popular. I could compare Apple to Inspiron which is even more popular and Apple would be even further ahead. Hopefully Dell can make some good laptops one day.

I've got a stacked 64Gb Dell XPS 5550 and an ass end M1 MacBook Air. I'd rather use the Air any day.

But I'd rather just use my desktop M1 Mini than both. It has a 27" screen and a proper keyboard (Durgod K320) attached to it. All my shit is in AWS now.

The M1 MacBook Pros are cheaper, because they replaced the cheapest Intel MacBook Pros in the lineup. Before M1, $1299 was the price of a MBP with 1.4GHz Quad-Core Processor with Turbo Boost up to 3.9GHz / 256GB Storage / Touch Bar and Touch ID: http://web.archive.org/web/20201105002246/https://www.apple....
feedback loop is longer, if my workflow is:

make change -> compile -> build image -> deploy to k8s

then running docker/k8s remotely would be great but i need great upload bandwidth to be pushing fresh images everytime I make a change. In theory docker layers/caching should fix this but the reality is that you don't add individual files to an image, you had the compiled artifact/bundle/whatevs and if any file changes its going to upload it again, made worse by a change that effects many components/images/k8s pods. If you can solve the upload bandwidth problem (local servers? giant pipe?) then becomes more possible.