Most people are not in private prisons (< 10% [0]), even if there shouldn't be any at all. Of course, there are still many "contractors" and "vendors" (phone service providers, food vendors, etc.) in public prisons which grift everyone.
After becoming familiar with the reality of the cost inflation of (in my case local government real estate) development projects vs private I chalked it up to graft, incentives, and mismanagement.
Actually your comment is probably more correct - adds a whole step to move the wallet. Misaligned incentives and mismanagement are probably more equal across public/private than we like to believe
I'm being a little bit facetious. When the government actually owns/operates the labor or equipment they can do a lot more. In the prison example state COs are certainly better than rent-a-cops.
It's just unfortunate that's how most administrators work. The traditional debate about public vs private usually focuses on different tradeoffs and incentives of the public - but if they are just paying market vendors it's greatly diminished.
I agree the officer story is the most significant difference, with state COs being more like police - likely to be well trained, have a long term stake in the career, and having somewhat of a social service culture.