[GFischer]

For whatever reason, software engineers don't have the luxury of doing a diagnostic.

That´s not necessarily true. I´ve worked with very good, serious consultancies, and I´ve seen a quote for thousands of dollars for exploratory work to give an estimate. Of course that was for a million dollar project - smaller projects might not have this opportunity.

Other companies had some interesting approaches where they gave order-of-magnitude estimates and then refined the estimate iteratively.