[jlouis]

You are missing that major-release compatibility is something which is taken seriously. Usually you have opcode stability for at least one release so OTP20 can run OTP19 bytecode. But not vice versa, naturally.

The same is true for serialization: new features are often introduced and then put to use a couple of major releases later. This ensures backwards compatibility. If you couldn't upgrade your cluster one node at a time, safely, then you would have to stop the system. The serialization format is also built to be machine-agnostic: You can run data from a 32bit little endian windows machine to a 64bit big endian sparc machine if you want, and it will work seamlessly. Of course that flexibility doesn't come for free and has an overhead. Another benefit is that data-at-rest can be safely decoded for every Erlang version back to at least release 6. This is quite useful in many situations.

So the path is usually to upgrade the OTP version first and then start using new features once the cluster is upgraded.

In the OTP20 release, a change happened in late RCs of the release because serialization stability was brought into jeopardy. It was reported by RabbitMQ and the serializer was changed so it is properly backwards compatible. Upgrade paths are important.