My favorite example of an 'ordinary' person who gained an extraordinary mental ability is Daniel Tammet. He learned Icelandic in a week. Spent 2 weeks and memorized pi to 22,000+ digits.
I really enjoyed reading about Daniel Tammet and his book (Born on a Blue Day) is fascinating, but I'd hesitate to describe him as 'ordinary'. He's likely somewhere on the autism spectrum and also has synesthesia which affects his perception of numbers (amongst other things); his relationship with the world is going to be very different than for most people.
He had one epileptic event in his childhood which may be why he was called 'ordinary.' The confound with autism of course makes the 'ordinary' designation hard to justify though (as you suggest).
I don't think it is correct to say that he memorized pi, more that he recited pi because his brain simply produced it for him he did not have to look it up.
Except that he wasn't actively calculating more digits of pi, some part of his brain had remembered the digits and was playing them back. Maybe some sub-conscious part memorized it, but it was still memorization.
So just to put my neuroscientist hat on for a moment, memorization is almost always used to mean the use of some mnemonic in order to enable later recall. I'm saying that he never looked at the 137,493rd digit of pi, and that could very well be wrong based on what actually happened, but I think of this more like the other savant in the video who just has the day of the week for a date pop into his head even for dates that have not occurred yet. In those cases 'memorization' is not the word to use.
There are many different ways to memorize information, including mnemonics, visualization, auditory recall, conceptualization, and probably a lot of others. Ancient Romans and Greeks used the method of loci[1] to memorize massive amounts of information that used spacial memory to store information in a virtual palace or room in your mind, which doesn't use mnemonics. From the wiki:
"In this technique the subject memorizes the layout of some building, or the arrangement of shops on a street, or any geographical entity which is composed of a number of discrete loci. When desiring to remember a set of items the subject 'walks' through these loci in their imagination and commits an item to each one by forming an image between the item and any feature of that locus. Retrieval of items is achieved by 'walking' through the loci, allowing the latter to activate the desired items."
Different aphasias like anomic aphasia(the inability or difficulty to retrieving particular verbs or nouns)[1] indicate that the systems used to store and retrieve different forms of memory differ depending on the type of information, or at very least that the concept of a thing is stored differently than its word.
All calendar calculation is done with techniques and/or memorization. All pi memorization is done with mnemonic techniques. I've been in that scene for 7+ years and have never seen a credible exception.