Is there a way to ensure that whatever happens (CPU, network overloaded etc) one can always ssh in? Like reserve a tiny bit of stuff to the ssh daemon?
Create a systemd override (by using systemctl edit sshd) and add MemoryMin=32M (or whatever makes sense for your system). This makes sure sshd is never pushed out into swap.
From time to time I’ll run something dumb on my machine (e.g. GC aggressive the wrong repo) and if I don’t catch the ramp up the machine will lock up until the oom killer find the right process. Sufficiently locked up, accessing alternate ttys to kill the offending process won’t work either.
I guess I could just reserve ssh then ssh into it from an other computer but…
On Linux I’ve done this by pinning processes to a certain range of CPU cores, and the scheduler will just keep one core free or something. Which allows whatever I need in terms of management to execute on that one core, including SSH orUI.
https://www.freedesktop.org/software/systemd/man/latest/syst...
You can also use sibling knobs to increase the CPU and IO weights of the unit, for example but setting this to something higher than 100:
https://www.freedesktop.org/software/systemd/man/latest/syst...