[adgjlsfhk1]

this is technically true but the Markov chain would be too big to store even with petabytes of storage.

[tgv]

Indeed. The argument boils down to: since it's finite, I can turn it into a FSA. Not only is that unhelpful, it doesn't tell you how to construct it, i.e. the learning process.