> Well, we know that it's impossible to remember everything.
Yet it's possible to remember a lot. Those who pass "The Knowledge" test are truly inspirational. See this 60 Minutes report on London's Black Cab drivers [0]
Remembering and learning are very different activities, however. It is pretty impressive what people can learn! And some of us do have superior knowledge of what appear to be "useless" trivia as well. (Ever seen Jeopardy?) But it is very well-researched and understood that _most_ of us have very bad memory and recall compared to a computer. One popular hypothesis is that every time a particular memory is recalled, it is slightly changed until it bears little resemblance to the original event, or even the earlier memories of it.
...yes but apparently while the posterior hippocampus actually grows in these people the anterior hippocampus shrinks... so there may be tradeoffs required to get this level of spatial knowledge... https://www.pnas.org/doi/10.1073/pnas.070039597