The GUI is usually the problem. I have booted Xubuntu and it's still slow. Slow systems with older GPUs simply can't keep up with newer desktops. Most of the times I need a terminal but a lightweight desktop can help if I quickly want to open a browser and search something so that I can copy past more complicated commands.
It fits into under 700 MB and runs in well under 100 MB of RAM. The default Ubuntu image is about 6 GB now and takes a gig of RAM.
Have you not tried it? I have:
https://www.theregister.com/2024/02/14/damn_small_linux_retu...