You're thinking too concretely. The hardware we know is only one tiny implementation of computation. The broader concept holds across any writeable tape of any sort, whether it uses voltages in silicon, rocks on a beach, or exists only in a person's mind, does not matter one bit -- they're all equally powerful.