[beagle3]

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable).

If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (even docerized and stuff), you are likely better off NOT upxing.

[FooBarWidget]

> and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable).

I don't think people care about that nowadays, seeing how popular Docker containers are. I think Docker containers already make it so that you cannot share executable memory between different containers because each one runs in its private namespace.

[beagle3]

Do namespaces separate file system caches? That would surprise me.

[ptman]

IIRC it depends on which storage backend for docker you use: https://web.archive.org/web/20170405122924/https://developer...

[viraptor]

The only one that could matter here would be mnt. And since bind mounts normally wouldn't cache pages separately, I don't think that would happen for the namespaces either. Happy to be proven wrong though.

More specifically I wouldn't expect "free -m" to produce different result depending on the namespace it's run in.

[smegel]

> you stop the OS from being able to page out your executable (rarely a big loss)

Why is this exactly?

[jlebar]

I think what OP meant is, if you have a page that's backed by disk, then when you're low on RAM, the OS can simply drop that page -- it's sitting there on disk if you ever need it back. But if you have a page that's not backed by disk, then you have to write it out to disk before you can drop it.

[beagle3]

Indeed, as jlebar notes - since the actual executable code (the thing that the CPU executes) does not exist directly in the executable file, the OS will have to write it out to the swap if it needs the memory (unlike uncompressed files, where it just reuses the memory and later reloads from the executable file).

It is rarely a big loss, because executables that are in use tend to remain in memory if the program is actually active. If you have a 300MB demon that sleeps, though, you will likely notice a swap out to magnetic disk.

[pgeorgi]

Also if you have the same executable running several times on a machine, they share the RAM for the code (read-only pages, which should be most of them).

That doesn't work for UPX because each execution decompresses anew, which makes it a "new executable" from the OS' point of view. The only thing that would help is kernel space merging, but that's really only activated for some virtual machines.

[icebraining]

that's really only activated for some virtual machines.

And even there, it's a security threat, since it enables cross-VM timing and other attacks.

[foobarchu]

I'm curious, why did you feel the need to call out go specifically? Does this apply more to go than other source languages?

[beagle3]

I wrote "or any other binaries, really". It's just that

(a) go compiles static binaries which makes UPX especially effective, unlike e.g. common C and C++ projects with their hundreds of .so/.dlls ; Delphi/FPC and Nim or the other two ecosystems that share this trait, but neither is as common as go.

(b) it's not good for non-static binaries, that is C#, Java, Python have no benefit from this.

(c) At the time I posted it, there were already 3 or 4 posts extolling the virtues of compressing Go executables.

[MichaelBurge]

Many go programmers statically compile their code, which should make the paging problem worse.

[takeda]

If you look at other responses large number of them actually mentions that UPX works really with go and docker, I believe it was response to that.