I still don't understand how consensus works off the chain. If 51% of the stackers just decide to run their own software and do their own thing. How does something off chain change that? How does something off chain penalize a majority of staked ETH?
If the 51% is a single staker or small group of colluding stakers attacking the chain, the rest of the users will probably not want to continue using that chain. The users can activate a soft fork, just by changing the rules of the code to burn the attacker’s funds, and running the updated client software. The remaining honest stakers can then continue to follow the head of the non-attacked chain, and the attacker would be the only one left on their chain.
Most crypto including Bitcoin is governed off-chain. People come to consensus on a set of rules that allows them to create a public and permissionless BFT ledger. Attacking the ledger then costs millions or billions of dollars. If one dishonest node or a group of colluding actors decide to spend this much to amass majority control of the network, the rest of the network can decide to fork the protocol to burn their staked capital.
The main goal of crypto - the base token and protocol rewards from PoW or PoS - is to secure this network and keep it permissionless and decentralized.