Unfortunately most packages spend more time during the configure step or other ~single threaded parts than actually compiling stuff in parallel when you have that many cores. You can compile multiple packages in parallel but doing that without running out of memory when you do get to a couple of big packages is tricky.
Ideally, portage would take memory (including ramdisk) requirements of packages into account when scheduling builds (some packages do check for free memory) and ask as a make jobserver [0] (with patches for non-make build systems to support jobservers) to always fully use the available resources.
Ideally, portage would take memory (including ramdisk) requirements of packages into account when scheduling builds (some packages do check for free memory) and ask as a make jobserver [0] (with patches for non-make build systems to support jobservers) to always fully use the available resources.
[0] https://www.gnu.org/software/make/manual/html_node/Job-Slots...