On most posix systems everything running as your UID has access to everything in your home directory.
I have no private key material or credentials in my .ssh folder (other than usernames and hostnames). All of my SSH private keys are stored in hardware.
Yes; if you dig into the links I left, Git can use a variety of credential managers to protect them.
Speaking of ssh key passwords: until OpenSSH 7.8 (2018-08-24), private keys using the PEM format were vulnerable to brute-force password cracking. You had to specify the -o option to use the more secure OpenSSH-format keys. Today the -o option is the default (and thus gone), but you might want to rotate your keys if they're from before September 2018.