HDMI isn't like composite video -- it is more than just a physical connection, it is a highly complex set of protocols with DRM and version numbers and everything.
I haven't run into Chromecast issues with my TV, but I have seen dozens of cases where two devices that ostensibly both supported "HDMI" refused to work together while the individual devices would work fine with (most) other devices. So I find it perfectly plausible that the OP's situation is very real even if I haven't experienced that particular one myself.
This isn't remotely limited to the chromecast though. The HDMI video signalling protocol is old and robust and works well and near-universally. The HDCP spec and CEC remote control features, on the other hand, are cesspools of complexity that work pretty much nowhere sanely.
I haven't run into Chromecast issues with my TV, but I have seen dozens of cases where two devices that ostensibly both supported "HDMI" refused to work together while the individual devices would work fine with (most) other devices. So I find it perfectly plausible that the OP's situation is very real even if I haven't experienced that particular one myself.