I am curious as to what class of problems are being solved on these super computers. Also whats the abstraction of computation here. Is it a container :-t :-t :-t
Weather modeling - X kilometers by Y layers of atmosphere can get expensive to compute really quick. And NOAA does more than just simulate weather, they're running climate/sea level rise/arctic ice modelling, aggregating sensor data from buoys/balloons/satellites, processing maps, and more.
I can't speak for NOAA, but my experience with supercomputing has been that there is no abstraction of computation, your workload is very much tied to hardware assumptions.
In my experience it's very hard to write code for parallel compute workloads and I am guessing that half of the code written would be creating abstractions about that.
They are used to do large-scale high-resolution analysis or simulation of complex systems in the physical world. The codes typically run on the bare metal with careful control of resource affinity, often C++ these days.
They aren't just used for global-scale geophysical processes like weather and climate or complex physics simulations. For example, oil companies rent time to analytically reconstruct the 3-dimensional structure of what's underneath the surface of the Earth from seismic recordings.
What do you mean bare metal? Because all of the big DOE computers are running Linux. Users will probably use hardware specific libraries (like CUDA/ROCm) and occasionally write some hardware specific asm, but none of the big computers are running without a POSIX OS.
I think they mean not in a VM, and instead using some job manager like slurm or condor. Typically users wouldn't have superuser privligles precluding the use of thigns like docker, which is why Singularity exists.
no, its a gang scheduled process - at least that's been the standard model. those processes are run either close to bare metal or as a process on linux. containers would be useful to package up the shared dependencies, so that may have changed.
I can't speak for NOAA, but my experience with supercomputing has been that there is no abstraction of computation, your workload is very much tied to hardware assumptions.