Hacker News new | ask | show | jobs
by hannob 3167 days ago
Surprised this still exists.

A little walk down memory lane: I once ran the exe mailing list for exe packers and protection tools. There was a whole scene of people in the 90s writing such tools and writing unpackers and removal tools for such things. UPX was one of the later ones that still existed when most of this scene vanished.

6 comments

The demoscene continues to make extensive use of packers, more for achieving even better compression than for protection against RE:

https://in4k.github.io/wiki/exe-packers-tweakers-and-linkers

(Incidentally, these advanced packers also tend to frustrate RE to some extent, since the same tricks they use to increase compression ratios can often greatly confuse RE tools.)

"RE" is reverse engineering, maybe?
I think some of that still exists, but the goal is to to evade anti-virus instead of compressing and deterring RE:

https://hackforums.net/forumdisplay.php?fid=299

Yes, this was my first experience with this piece of software. You can pretty clearly tell that it is from UPX by examining the file in a hex editor.

I still have the malicious file on VM for me to do some analysis on it later. (if anyone would like it, feel free to contact me) edit: added the contact me

I remember cases where the AV successfully detected the upxed executable, but not the original, because upx was so widespread that the most common version of the infected file was upxed.
"Packed" executables are also still used in the demoscene for fitting into size limit categories.
Exists and maintained. They even keep adding support for more formats. That's pretty amazing after almost 20 years.
I use it for static binaries I use where I don't care about startup times being slower.

You'd be suprised at how much of an elf binary is all 0's.

Compressing them this way however makes situation worse for memory manager.

If you use uncompressed (or transparently compressed by the filesystem) binary, your process has mmaped the memory pages, which can be discarded and then reloaded, as needed.

If you use self-extractor, your process has dirty pages that it itself wrote, that cannot be discarded, but must be moved to swap if needed.

The more you use the same executable for multiple processes, the worse the effect is. The ro mmaped pages are shared among them all, the written pages are private to each process.

For the binaries used this isn't really a concern. Most are cli binaries run every so often.
How large are those binaries?

I would be surprised to see practical performance degradation in uncompressing executable code before jumping to the program on today's machine. The largest binary in my /usr/bin/ is 50 megabytes. On the other hand, for very, very large binaries it's probably faster to decompress in memory rather than load all the bits from disk.

Further, most executables aren't static these days. (I often wish they were, though!). What type of binaries have you got, and are they really so big that it's worth the hassle to compress them just to save disk space?

Just interested.

50MiB to 6MiB

The binaries are mostly stuff like pandoc and compiled statically so that I can run them anywhere. Nothing too special.

Its not technically needed, but it makes network transfer faster and in general thats good enough. Its not really intended to reduce disk space really, just more a way to make things more manageable.

Curious — which network transfer protocol are you using that doesn’t support compression on the fly? And if its just for transfer, why not gzip instead?
UPX is usually better than other generic compression utilities. IIRC it actually modifies the executable before compressing it to provide better compression. I think on Windows it reversibly changes relative addresses to absolute addresses so you get better compression rates.
2GiB quotas on home directories is another reason, but explaining it all on hn isn't really a goal of mine. Suffice it to say there are a confluence of issues. As to network compression via ssh say versus compression of the binary, time it you might be surprised at the difference.
Not only does it exist, but it's insanely good at packing go static binaries. I don't remember the compression ratio, but I think it's something like 20% of the original size.
In the 90s I was still starting to explore computers and executables, etc, and it blew my mind when I found about packers and protection tools and RE in general. Good times! Oh nostalgia...

I just remembered: ProcDump32! Geez, that really blew my mind at the time and in a way still does.

Modern DRMs and protectors are based on the same concept(s)/protector(s) as the exes packers from the nineties.