The X32 ABI is interesting. Who'd have thought this many years after 64 bit becoming mainstream, (and decades after it becoming mainstream on servers) something like this would be needed.
Anyone has a good argument why NOT to use this for every non memory intensive application? I always thought that using 64bit pointers was a waste of memory when not needed and this x32 abi to me seems fantastic since it keeps the several advantages of newer 64bit cpus (more registers for example as mentioned in the article).
Recently there was a HN thread [1] about the conservative garbage collection used in some 32bit VMs (the complaint started with Go), because the way their GC works is by scanning portions of process memory for 32 bit values that map to valid address space for that process. Those values are treated as pointers even if they're not pointers, so the data they point to is assumed to be used and is not garbage collected.
Architectures/ABIs with 64bit pointers have a much lower probability of arbitrary values mapping to active memory space for the process, so memory bloat caused by the conservative GC strategy is much more limited.
The x32 abi looks like it would suffer from this 32bit conservative garbage collection problem too. Short of changing the VM strategy, using 64 bit pointers is the only way to ensure that such VMs do not accumulate so much unused but not-garbage-collectable memory allocations.
An application that uses 64-bit pointers but doesn't need 64-bits of virtual address space is wasting physical memory because your pointers are twice as large. The application would waste physical memory pages and memory bus traffic just to store 64-bit pointers with lots of unused zero bits like 0x00000000ffffffff.
'not memory intensive' is an ambiguous term. he meant 'does not use a lot of memory, but may access what memory it does use quite a bit'. you meant 'does not access memory often, and performance thus does not depend much on memory access latency'.
seems like a great idea. toolchain support may be an issue for a while, especially if you link against libraries with a different ABI. one thing to keep in mind is that the performance gain from e.g. more registers may not be as big as you'd think, since the hardware already plays lots of tricks to get around i386's limitations (e.g. register renaming) and likely extracts a good bit of the available ILP.
Actually main line kernel support is one of the last things that's been needed to get this going. gcc 4.7, and the latest glibc and binutils have had support for a little while. Now it'll still be a while before distros start supporting it as a first class citizen but the pieces are now there.
Well, Donald Knuth for one. Here's a blog entry I wrote about Donald Knuth and how he asked for this a while back. I think it makes a lot of sense and I'm really glad to see this maturing!
SGI's IRIX always had the ability to do since decades ago. Of course, like others have stated, the main reason is to conserve both memory (for pointers) and more importantly bus bandwidth (which is a mayor problem in MPI/SMP systems memory, especially in those IRIX supported with 128+ cpus in a NUMA configuration).
There are still many features where linux is just playing catch-up that commercial unix kernels had decades ago.
In fairness, x86-64 Linux has supported running 32-bit x86 processes from the start (likewise for PPC and PPC64). It just so happens that the difference between 32-bit and 64-bit CPU modes is quite significant on x86. So this new ABI for using 32 bit pointers while the CPU is in 64-bit mode is really Intel/AMD's "fault" more than Linux's - the 32-bit mode's register starvation is particularly crippling. (Actually, if anyone is to blame, it's Microsoft "fault" for tying consumer computing down to x86)
IRIX may have features that Linux lacks but this is not an example of one. Linux supports 32-bit binaries with 32-bit pointers that run on 64-bit kernels on the other 64-bit archs, like powerpc or sparc64.
This problem is really a quirk that results from the fact that AMD made significant changes to the arch(added 16 registers) when they invented amd64.
Is the X32 ABI meant so that one can run X32 programs on an x64 kernel as well, or does one have to decide to use X32 once and for all and then compile everything with this ABI?
My understanding is that the X32 ABI exists only in x86-64 kernels, and requires relevant X32 userspace libraries.
Basically you can choose if you want your program to have only 4 GB address space and thus only consume 4 bytes of memory per pointer. This is a per-process choice. (but obviously requires special binaries)