I actually back up my Gradle cache in particular because things have a tendency to disappear from the internet sometimes, and permanently break builds of older stuff.
The build dependencies are the problem. The build process downloads them from hardcoded internet URLs and they are cached for 30 days. By backing up the cache, I can (probably) rummage around it later to find anything that's been deleted from the internet.
Otherwise, you're directly heading for a nasty surprise at the worst moment.