Hacker News new | ask | show | jobs
by njs12345 3158 days ago
CPU also causes a similar problem - for instance Java (and lots of runtime Java libraries) use the number of available CPUs to decide how many threads to use for various tasks, which can cause terrible performance if you're giving a container 1 CPU of a 48 CPU server and it thinks all 48 are available - we encountered this at work recently and eventually bodged around it, but definitely a gotcha that this stuff isn't handled properly by runtimes yet.
4 comments
dogruck 3158 days ago
Anyone who is running important production java programs without carefully considering and testing the JVM’s command line arguments is headed for trouble.
link
the8472 3158 days ago
java9/hotspot has -XX:+UseCGroupMemoryLimitForHeap to deal with the memory part and I think the method to get the number of CPUs also takes cgroups/taskset into account now.
link
praseodym 3158 days ago
Actually the `-XX:+UseCGroupMemoryLimitForHeap` option is also in Java 8 since Java 8u131, so it's available for nearly everyone right now. It doesn't solve a lot of problems though: by default the JVM will use only 1/4th of the memory (cgroup) limit as its heap size, which wastes most of the container's memory. It's thus attractive to use `-XX:+UseCGroupMemoryLimitForHeap` together with `-XX:MaxRAMFraction=1` to get maximum heap size given your container limits. However, because total JVM memory is heap+permgen+threads*stacksize, you'll leave no memory for threads this way. If you're running an average web server, it will spawn dozens of threads and get your application above the cgroup memory limit (=OOM killed) pretty fast.

The only way out is to configure memory limits by hand, taking into account the maximum number of threads your application will use. That usually isn't trivial.

link
the8472 3157 days ago
They're changing MaxRAMFraction to take non-integer values, so in some upcoming java release (yay 6month release cycle) that problem should go away too.
link
dominotw 3158 days ago
> if you're giving a container 1 CPU of a 48 CPU server and it thinks all 48 are available

Are java processes usually deployed in (1 server to 1 java process ) configuration ?

Seems a bit strange for runtime to assume certain deployment configurations.

link
XorNot 3158 days ago
CPU works differently though. A server with little load really can give you all the CPUs unless you're pinning workloads (which most people aren't).
link