If you add more than a handful of plugins in any of these IDE layers for Neovim, like LunarVim, LazyVim, AstroNvim, &c., it can slow things down to an unacceptable level quickly.
No need to rely on your terminal to provide performance. LunarVim is very slow even though I use Alacritty, it was the main reason I stopped using it and switched to LazyVim.
If I'm being quite honest, I think LazyVim is the more polished project. I may end up switching to that; I've been impressed by what I've seen so far (as well as using the lazy.nvim package manager).