The argument doesn't rely on infinite memory.
In real systems external inputs provide an infinite source of "memory" to read from.