I don't see how you can run a minecraft server on those specs. Just turning the draw distance will cause problems. Add a couple users and you are fucked.
It's fairly trivial to overclock Raspberry PI 4 (Model B) to ~2GHz if you have active cooling, and with a single one I've run Minecraft servers for ~5 players without hiccups. Of course it's not gonna be able to host large servers as the specs are low, but for the price, it works out well for small friend-groups.
It's hard! I only had two users consistently (myself and a friend, and occasionally guests but they taxed the system a lot so it wasn't very often), but I still even up having to fiddle around with a lot.
* As someone else said, overclocking helped, and I had a reasonable passive cooling case to help there.
* I used Paper instead of the normal Minecraft server, and I ended up spending a decent amount of time optimising the configuration. Paper by default comes with a bunch of optimisations, I enabled some more, although I also disabled some that were interfering with the more technical areas of Minecraft that I enjoy more.
* Whenever things started lagging, I went on a killing spree for our main farms, and that tended to work well enough. Most of our contraptions were turned off by default, or designed not to be too laggy. I also restarted the server every night, which worked reasonably well as a sort of ultimate GC.
If I were going to do it again more seriously, I'd probably get a cheap mini PC and use that instead, but for what it was - me and a friend rediscovering Minecraft after having not played it probably in about 5-10 years - the pi4 held up pretty damn well.