I really wish more systems would adopt the "copyindent" pattern, where indentation is simply copied from the existing line. I've primarily developed in vim for 15 years and have similarly minimal `vimrc`s (no plugins), but some of the first configurations are always:
set autoindent
set copyindent
set preserveindent
This should be compatible with basically any language (other than some esolangs like "whitespace") and any tab/spaces scheme. There's no need to have per-file/project configuration, as long as you're okay with pressing space 4 times instead of tab in the projects that use 4-space indentation.
The ftplugin for the make filetype includes setlocal noexpandtab, which should take precedence over what's in your vimrc.
% vim Makefile
:verbose set expandtab?
noexpandtab
Last set from /usr/share/vim/vim90/ftplugin/make.vim line 15
% vim file.c
:verbose set expandtab?
noexpandtab
Last set from ~/.vim/vimrc line 56
but I think that may not work as intended for OP as they don't have "filetype on plugin".
It surely can. If I have to casually mess with a Makefile, I'll just remember to use control-v TAB as needed, or vi yank/put another line with a proper tab and edit. In the very rare case I'm doing more serious work, I'll temporarily turn it off.
You can change it per filetype. For example, I have a similarly minimal vimrc that defaults to sw=4, ts=4 and expandtab, but with the following additions:
autocmd FileType go,make,sh set noexpandtab
autocmd FileType js,json set sw=2 ts=2
Edit: I think you might also need the following to make it trigger the FileType event when you open a file in the first place: