This was taken to the extreme in the novel Accelerando. The main character is mugged and loses his "cyberware". He thus loses the memories and thinking capabilities he had offloaded to the hardware.
Considering how stunted most people are without a phone and google on hand. Sounds like the main character had a rough time learning to spell and do math again :).