Even if remembered perfectly and consisting only of valuable information, it's amount of knowlege that fits in a single brain. We are centuries past the point when such amount was significant.
You could distribute specialized knowledge among different people, so it's not as restrictive as a single brain. And to be fair, our most valuable knowledge today is probably encoded in few people's brain as well in the form of institutional knowledge. But I get your sentiment, writing down is an immense construction that expands the topology of knowledge by a huge factor.