I've tried many times to learn Haskell. The problem is always with setting up a compiler, getting packages, setting environment, not getting a ton of gigabytes of duplicated packages, getting useful vim support etc.
Dependencies are downloaded and compiled once for all your projects using the same Stackage release and are shared between projects.
In the end you still might have duplicated dependencies if you use different stackage.org releases but it's a much better situation than using Cabal sandboxes where every project has its own copy of dependencies.
You build against a fixed set of packages defined by "Stackage" so that you only need to build them all once. While this set evolves over time, if many (or all?) of your projects target the same Stackage LTS then you won't need to rebuild dependencies.