[ruchir]

I think Turing's key insight was that a special FA with a tape mechanism could be devised that could imitate any other FA. The power of this FA in imitating other FAs is limited only by the length of the tape.

However the improvement of this construction over any specific FA is that you have a device that can run any FA. As the length of the tape increases you have a device that can imitate the any and all possible FAs.

The CPU is precisely this FA which can imitate any other FA. The tape is memory.

However it is important to note that, at the end of it, any specific program that is being run on finite memory is still an FA.