I could cover the entire of her latest album and sell copies? I didn't think copyright allowed for that without permission but maybe I don't know copyright like I thought I did.
Yup. As mentioned, she doesn’t own the rights to her early albums, so this year she actually re-recorded and released alternate versions of her own music.
Cover songs are covered under “compulsory” licenses. You can safely cover an entire album without permission as long as you pay the royalties to the writers.
The entire “songbook” genre is basically this. If you want to drill down further there is an entire category of albums which is just people covering “Dark Side of the Moon”