[arc0]

The busy beaver function assumes infinite memory, it is defined as a function of the number of states of the Turing machine (in a programming language this is program size). If it was memory then it would not be an undecidable sequence.

[lmilcin]

In general, when you want to implement some kind of algorithm in real life the state of the Turing machine must be stored somewhere between ticks and this storage can be considered memory.

You could think about this way: hardcoding a constant (moving it from heap to compiled code) doesn't magically cause the program to use less memory.

Thinking it in a different way, from purely physical point of view, every bit of information can be translated to some minimum amount of energy or mass (mass energy equivalence).

Since state ("state", not "possible states") of Turing machine is bits of information, you can't have a Turing machine with infinite size of state. Now, "possible states" are capped because if the state is finite in size, the number of possible states is less or equal than all permutations of it (permutations == possible states).

It is largely philosophical question which is "memory" available to the program and which is "Turing machine state". In case of real programs we see that memory can be reassigned depending on requirements.

There are some cases when the distinction becomes important in reality. Consider a trained AI that gets "baked in" and shipped to the user to compress/decompress images.

Let's say it does fantastic job at compression and decompression but takes 2GB of disk storage and memory when executing.

If you just compress a single small image you realistically need to send 2GB of the AI plus the very small image, but when you have billions of images this static cost of the AI gets amortized.

The same way happens when you run any real program on an operating system. In reality the program is much larger because even if it prints Hello World it still needs to do a bunch of data like recognize your monitor it is talking to. We conveniently package the common parts of the program as "Operating System" and then just let exchanging the small part that will make sense when coupled with correct OS.

That's also how you can have very small webpage that takes GBs of memory to execute... sadly...