[white-flame]

We deploy distributed, multi-language, centrally Lisp/SBCL servers as well. A few specifics that I'd point out:

Many of SBCL's optimizations are fine grained selectable, using internal SB-* declarations. I know I was at least able to turn off all optimizations for debug/disasm clarity, while specifically enabling tail recursion so that our main loop wouldn't blow up the stack in that build configuration. These aren't in the main documentation; I asked in the #sbcl IRC channel on FreeNode.

You can directly set the size of the nursery with sb-ext:bytes-consed-between-gcs, as opposed to overprovisioning the heap to influence the nursery size. While we've run in the 8-24GB heap ranges depending on deployment, a minimum nursery size of 1GB seems to give us the best performance as well. We're looking at much larger heap sizes now, so who knows what will work best.

While we haven't hit heap exhaustion conditions during compilation, we did hit multi-minute compilation lags for large macros (18,000 LoC from a first-level expansion). That was a reported performance bug in SBCL and has been fixed a while back. Since the Debian upstream for SBCL lags the official releases quite a bit, it's always a manual job to fetch the latest versions, but quite worth it.

Great read, and really familiar. :-)

[vseloved]

We played with sb-ext:bytes-consed-between-gcs, but couldn't find the right balance. That's why we were surprised with the result of the oversized heap experiment

[white-flame]

You might want to do some googling about it, but I also seem to remember that SBCL might decide to use large or small MMU page granularity depending on the size of the heap. That might be the watershed trigger for performance. (or just some random mis-remembered nonsense)

[vseloved]

Can you also point to the particular version of SBCL that fixed for long-running compilation? We have recently upgraded to one of the latest version, but I think I've missed this change - I'm interested to check it in more detail.

[white-flame]

I think it was back in the 0.54-0.59 version range.

[vseloved]

that's much older than what we used recently, so it should be a different thing