[andrew_v4]

It's actually "accountability" that's a big part of the problem.

Government procurement is so focused on the appearance of fairness and money saving that all other goals, like actually getting something that works, take a back seat.

You end up with over-specified requirements that remove the possibility of innovative or creative solutions. Providers are treated like a commodity, where it is assumed that all will do the same job, and cost is the only real negotiation point, maybe with some kind of scoring grid against the over-specified requirements thrown in.

And the procurement decisions are made by procurement officers who are not the actual users of what is being bought (in the name of objectivity).

So what happens, on a good day, is that the operational users in the purchasing department work with the preferred vendor to "wire" the RFP to reflect the scope or work that is wanted and add requirements (e.g. years of very specific experience, past projects) that heavily favor the preferred vendor. At least this way the department may get something they want, thought it obviously can be gamed. Worse though is that many contracts just go to lowest cost staffing firms that are optimized to comply with government procurement requirements and provide the minimum set of bodies that meet those requirements, usually former government folks rented back, plus some low cost IT resources, that are there to execute to the letter of what the government has over-specified, usually something that wont actually work as written.

This is why so much government procurement is a failure by any objective measure. What I have seen work is when a vendor provides a credible unsolicited pitch to a known problem at a fixed cost, and the relevant departments are forced to decide if it makes sense.

In Canada we had a major one like that a few years ago, the outcome was great for the department that needed it, but careers were destroyed in the process as politicians and their incumbent friends pushed back to try and stop it.

[crazygringo]

This is exactly it.

And to be clear, there's a good reason for it: it's to prevent corruption.

If things aren't overspecified and providers aren't treated like a commodity, then it's incredibly hard to prove that a government official actually awarded a contract in a fair process, rather than just sending it over to their best friend's business.

Unfortunately, nobody's really come up with any reliable process for having the flexibility to get good products for good value, while reliably preventing corruption. And when there aren't these ironclad protections against corruption, experience shows it turns endemic, so much money flows through the government.

It's a seriously tough problem.

The reason it doesn't exist in the private sector is that the chain of accountability from managers to CEO to board seats is actually quite strong, and shareholders are incredibly motivated to extract profits. The accountability to voters in a democracy, on the otherhand, is far, far, far weaker -- as voters vote primarily along party lines or on only the absolute biggest hot-button issues.

[BadInformatics]

I'm skeptical it's even good at that intended purpose. Perhaps one could argue it prevents blatant, direct corruption, but it does little to control for large company influence and other forms of soft power.

The biggest companies in this space maintain an active revolving door, which ensures that procurement policy is moulded (either consciously or unconsciously) to their process and needs over time. Even more insidiously, they've convinced governments to gut their own IT workforce, removing the people most qualified to critically analyze software vendors. This appeals to your average bureaucrat because it appears to strike a good balance between effort and risk minimization (e.g. why bother managing multiple smaller vendors or timelines?), while in practice it does exactly the opposite.

[jq-r]

Yeah, I'm not sure if it prevents corruption at all. In my country public tenders are just another word for corruption.

A real example: police force wanted to get say 1000 new squad cars. One of the points in the tender was that the car's trunk has to be exactly that many litres (say 307L, don't remember the exact number). So of course, only one model of all the cars from all manufacturers had that value, and of course the only dealer who submitted for that tender won it. So it was blatantly obvious that the process was rotten from the start. But it was legal. And they (government)did it many times. And pretty much they are doing it for the last 20 years or so. So corruption is not something which you can solve easily, you need a lot of checks and balances to make it work.

[crazygringo]

You're right in that it absolutely requires either a watchdog agency to ensure tenders are written in a neutral way before being issued, and/or a court system where losing bidders are able to successfully sue as soon as they're issued, on the grounds of the tender not being neutral.

In one country where I previously lived, there was also an "escape clause" where if there was emergency time pressure, you could circumvent the process -- so guess what? The government would "invent delays" in writing up the specifications until the last possible minute, then award the contract without a public tender because there was no time left for the tender process!

So yes, the process absolutely has to be designed with some form of oversight and without loopholes, in order to achieve the aim of preventing corruption.

[crazygringo]

It's to avoid the very specific form of corruption via kickbacks, essentially -- and if you look at the history of how politicians used to spend money in the US, you'll see that it is actually quite effective at this, and that it was once a gigantic problem -- and continues to be in some countries today.

You're right that it does nothing about other forms of influence like the revolving door.

And like I said, it comes at a tremendous cost of efficiency and quality. It's not trying to strike a balance between corruption and efficiency/quality, it's trying to explicitly minimize corruption at the expense of efficiency/quality.

[hedora]

> And to be clear, there's a good reason for it: it's to prevent corruption.

There’s consensus in this thread that this process reinforces corruption, and is controlled by the corrupt, so preventing corruption isn’t the real reason.

If you want to prevent corruption, hire engineers directly at market rate, and promote / retain them based on their ability to deliver projects.

If you want proof that this approach works, just study history. The US government used to work this way (back when our middle class and economic clout were growing) but corrupt politicians decided to outsource everything to increase the supply of kickbacks.

[crazygringo]

> There’s consensus in this thread that this process reinforces corruption

There is no such consensus, and the idea that it reinforces corruption is contrary to common sense -- it's self-evident that maximum corruption bypasses specifications and public bidding altogether and just hands a contract to a politician's friend.

> If you want to prevent corruption, hire engineers directly at market rate

It's not feasible for a government to accomplish all its tasks by hiring and never by contracting. It would be incredibly wasteful because many projects are one-off, whether building a new suspension bridge or a huge new IT project. It's like saying a company should have no suppliers and write all its own software from scratch.

> just study history. The US government used to work this way

I've studied quite a bit of history thanks, and government in the US used to be quite corrupt compared to today -- just look up Tammany Hall [1] if you'd like a quick introduction. Corruption in the US has very much decreased over the past 150 years.

Outsourcing has existed as long as government has existed. I think you're confusing outsourcing as a general concept with privatization as a specific issue, which is about one-off decisions to choose to start outsourcing things that were previously done in-house. Which has its own set of pros and cons.

But no government can in-house everything. So hiring in-house is not the answer to corruption.

[1] https://en.wikipedia.org/wiki/Tammany_Hall

[g_p]

> Unfortunately, nobody's really come up with any reliable process for having the flexibility to get good products for good value, while reliably preventing corruption.

I've seen one approach work, but it struggles to scale, as it needs technical people on the client side.

Buying "outcomes" rather than services can work well - rather than procuring a specific "specification", you buy a solution. The standard contractual framework means you are paid for delivery to tangible milestones (demonstrable value), with engineering/technical background project management team overseeing the work. You work at risk, as you only get paid for delivery. That keeps many of the charlatans away, since it's very clear you're paid for delivery, not effort. That means the headline rate is higher, of course.

Focusing aggressively on actual delivery, but also not dictating the solution means you can see suppliers compete not only on price, but also on how they'll solve the problem. This means the government client needs to understand their problem well enough to articulate it (with some of that support from a technical project manager), but they then evaluate proposals for solving their problem. This moves away from the incentives to "body-shop" low-pay graduates onto a project that a partner pitched for, as it has to actually deliver.

I tend to see the "worst" projects (in terms of non-delivery, large bills incurred, poor value, and the only output being a report recommending more work) come about when the government client doesn't understand their own problem or goal though, so perhaps this approach self-selects problems where the customer can actually articulate their need.

[rossmohax]

Problem with that is government is worst customer: don't know what they want, what they have and how to get anything done. It is very hard for companies to commit to deliverables, when incompetent department they need to integrate with doesn't play the ball.

[g_p]

Indeed, however this approach effectively offers them a carrot - if you understand your problem, you can use this (very effective and well regarded) route to getting your problem fixed.

The end result, as you'd expect, is mission-focused solutions to problems with minimal external dependencies. That means the problem gets solved in the simplest way possible, with the least overlap with incompetency possible.

It doesn't work for every problem, but it does show that forcing government to understand the problem before spending money can actually work, at least at some scale.

[creato]

> The reason it doesn't exist in the private sector

The same problem absolutely does exist in the private sector. Many of the same big government contractors are running almost the same scam on big companies.

I think this problem is more a function of the size of the organization than public vs. private.

[crazygringo]

I was referring to the problem of corruption and kickbacks.

I'm certainly not saying companies can't overpay for things, but there is an inherent pressure from competition to incentivize companies to try to pay less, whereas taxpayer-funded government often doesn't experience similarly direct pressure.

[anticensor]

Corrupt businessmen are a thing.

[crazygringo]

When I said it didn't exist, I meant it doesn't exist as a problem at the same scale.

Yes there are obviously employees who embezzle or contractors who defraud, but companies are generally able to police this themselves relatively effectively. It's nothing like government left to its own devices.

[briffle]

> Government procurement is so focused on the appearance of fairness and money saving that all other goals, like actually getting something that works, take a back seat.

I worked at a small 2 year college for many years. One time, my Dean I reported to was on vacation, so I had to go talk to the college president, and get him to sign a form for a $7 petty cash reimbursement for some zip ties I had bought to clean up some cabling.

One year, our President had to travel to the capital city (about 250 miles away, over the mountains) almost every other week for some budget discussions with other colleges, legislators, etc. We could have saved the taxpayers THOUSANDS of dollars by renting a modest house to use for him (and some of the other staff members that regularly traveled to the capital). But that "might" look like we were providing them with a second home, so we spent thousands more on hotels.

[tomc1985]

A house? A modest apartment I can see, but a house seems a bit much

[bsder]

Renting a furnished apartment can sometimes be almost as much as a renting a furnished house depending upon the area.

[metanonsense]

Last week we lost a bid for a government contract. That's nothing unusual but I almost laughed when they described how they reached that conclusion. They weighted price against quality at a ratio of 80 to 20. I mean: really?

[g_p]

This is fairly standard, sadly, and is why Government struggles to deliver, especially on IT and similar "intangibles" type contracts.

The same issues happen in any other procurement activity that is required to rigorously follow a specific process due to spending public money, or bill-payer money of a regulated monopoly etc.

In short, you need large numbers of people involved to avoid "corruption" (irrespective of the actual level of such risk), and this means you end up less flexible and less able to buy what's needed. Weighting price by 80% is common, as nobody wants to be seen to deliver "poor value for money to the tax-payer". Hence the cheapest bid almost always wins, as nobody wants to have to stand up and explain why they didn't pick the cheapest bid.

There's a whole separate issue in how to handle "too cheap" bids (i.e. where you under-bid on the initial work, knowing you can get technical lock-in and be able to win future contracts uncontested, and turn those lucrative), but this is still an issue - see how the large outsourcers or consultancies do this regularly, and end up winning renewals on basis of "necessity".

There's an art to writing a winning (cheap) tender, then staffing it with people who rigorously enforce the scope back onto the Government client, and force every single change through an expensive change process. That's the business model many follow, and it delivers far poorer value for money in the long run. But the headline price was cheaper, so they'll still get selected...

[xwolfi]

In France, the tiny company I was in lost a lot of gov contracts to our absolute surprise since we felt we actually had a better solution for the price.

What we did to start winning was to make friends with the people judging us, offering free services making them personally look good until we started having such relations with them they d ask us out to frame the contracts and give them to us whatever our competitors would come up with.

It's impossible to take decisions based on surprise proposals in a public tender and it felt it was an open secret that tenders' winners MUST be decided before publication.

[BadInformatics]

Name and shame: https://www.cbc.ca/news/canada/ottawa/phoenix-costs-137-mill...

[jjk166]

This is the best explanation for the phenomenon I've ever heard, thank you