I hadn't thought about the infinities part. If they had the same behavior in all operations involving other numbers, then I could see how they could nonetheless be the same from a C++ type perspective, but given they behave differently, C++ certainly couldn't consider them the same.
If you keep infinities and NaNs out of a subsystem, then float and double are totally ordered within that subsystem. So, it is a choice. I would not hesitate on that basis.
Of more moment is that floating point variables often represent imprecise values, such that e.g. 1.41 could represent a value that should equal another transcribed as 1.39. The type system will not help you there. You are obliged to code in tolerance the hard way.
Thanks for clearing that up!