I think they decided to play legal chicken and hoped Apple would blink, and Apple did not. The play wasn't entirely unfounded.
A lot of current law around things like app stores is existing in the liminal space of "Are these restrictions legal? Nobody knows. Do you want to be the one who stakes all your company's money and your investors' money and your entire business model on finding out?"
Epic did the calculus and seemed to conclude they wanted to be the ones to take the hit (possibly because since they also own their own app store, the math was a little win-win: any lock-in the courts find Apple has applies to the way Epic runs their store also). It hasn't worked out the way they'd hoped (though the anti-steering decision was interesting and a little surprising).
The blinking would have been because Apple does make a ton of money in the App Store. So it would have been cheaper for them, hypothetically, to pay off Epic to go away or to make some nonsense carve-out in their TOS that was hyper-focused on allowing Fortnite to work while excluding all other stores-in-stores than to risk the Courts would say "Hmmm..... Y'know..... The way you run your store looks... a lot like a monopoly. Maybe you can't run it like that anymore. In fact... Maybe you can't run it at all. Maybe no company that makes the hardware also gets to dictate the software that runs on that hardware exclusively. How about we give you, oh, six months to divest the App Store into a separate company and re-tool the iPhone OS to allow loading multiple stores?"
Perhaps not a likely scenario, but US law is highly path-dependent and at the end of the day, judges' interpretations of law and precedent are all that matters regarding the correctness of a court decision.
Epic's main mistake was performing the stunt of violating the contract and having a lawsuit ready for as soon as Apple suspended their developer account. Given they explicitly broke it as a means of creating a PR spectacle, the judge punished them by upholding Apple's suspension of their accounts (other than Unreal Engine).
If they had just sued without changing the iOS app to allow in-app purchases without using Apple's IAP, then Apple would likely have been punished if they still suspended Epic's account.
Well, they certainly did that. But how many billions of dollars in mobile revenue did they lose once Apple and Google dropped their game from the app stores? Pretty expensive PR exercise.
On top of getting slapped with a judgment for their own exploitative practices. Lots of poor decision making going on at Epic, well deserved decisions all around.
It's possible that this at least had some influence on the EU's decision to allow sideloading though (it gave publicity to this topic), so from that perspective it's not sunk cost.
A lot of current law around things like app stores is existing in the liminal space of "Are these restrictions legal? Nobody knows. Do you want to be the one who stakes all your company's money and your investors' money and your entire business model on finding out?"
Epic did the calculus and seemed to conclude they wanted to be the ones to take the hit (possibly because since they also own their own app store, the math was a little win-win: any lock-in the courts find Apple has applies to the way Epic runs their store also). It hasn't worked out the way they'd hoped (though the anti-steering decision was interesting and a little surprising).