[epec254]

The key part IMO is buried in the article - there happened to be an existing, perfectly accurate database containing all the required info about each employee - the same info that previously had to be manually found for each retirement.

Without this, this effort would not have been possible.

> Fortunately, we stumbled on a critical clue. While poring over old documentation, we discovered that OPM actually had data warehouses that stored historic information about every federal employee. Apparently, these warehouses were created as part of a modernization effort in 2007, and HR and payroll offices all across government have supposedly been regularly reporting into it.

> For some reason however, this was not well known at OPM, and those that knew about it didn’t know what data it held, nor considered how it could be used to simplify retirement processing. Not many had seen the data, and administrators were initially resistant to sharing access.

> From a software perspective, this was the holy grail: a single source of truth that held all the information that the manual redundant steps were meant to review. Because the information was regularly reported by HR and payroll, by the time an employee retired, OPM should already have everything needed to process the retirement, without anyone re-entering or re-verifying information.

[tigerBL00D]

Yes! Whoever built the data warehouses and keeps the data pipelines running would seem to be the real heros of this story. I sure hope that group did not get gutted by DOGE.

[xp84]

If the people who set up the DW and set up the data to flow into it did not also build any applications to actually take advantage of the data, they didn’t complete their job properly. It’s been 18 years, that’s long enough to document the existence of it. Some might say 18 years is even enough time to build at least one useful application powered by it.

I’m sure in reality the people who built this system were smart, and wanted people to use it, but were just buried under layers of technology-unaware management and bureaucrats who felt threatened, afraid it would marginalize or eliminate their paper-pushing jobs. But this very likely reality is just more proof that the government needs significant restructuring. Most people in management at the government are there purely because of tenure, not because they’re great leaders, nor subject matter experts in how complex things are efficiently built and run outside the government world.

[fn-mote]

> If the people who set up the DW and set up the data to flow into it did not also build any applications […] they didn’t complete their job properly

1. That’s a whole extra level of responsibility / management / bureaucracy. At some point, somebody near the top needs to care or it doesn’t all get done. The existence of this DB says somebody cared, they just didn’t have enough power.

2. I’m curious how this compares to experiences at Big Old Corp, like IBM or GM, not just the SV darlings.

[master_crab]

Thanks for pointing this out. I think it does a good job of also highlighting that most problems aren’t technical; they are either people or organizational.

[fnordpiglet]

Yeah this stuck out at me - the hubris of the stateless web stack supersedes the 18 years of hard unsung work at building and end to end stateful pipeline that ties out to the penny and handles all the complex business logic and reconciliations seamlessly across god knows how many integrations. No fancy diagrams or pictures of the nameless faceless heroes that had accomplished that act of heroism. For sure recognizing the value is something to trumpet, but that’s the Herculean hero story I want to hear - the DOGE bros who tied it all together with JavaScript frameworks, yawn.

[xp84]

The only way the database could be harnessed to do something useful is after all the people who were standing in the way in management for the last 18 years likely having been sacked. You can bet any useful project to put it to use was blocked by paper-pushers threatened by the spectre of automation, until most people had forgotten about it.

Nobody believes the database sprung forth from the earth or was created accidentally. The fact that 18 years later that project had borne no visible fruit, and that most people who could have used it, didn’t even know about it, is proof of the problem. It’s a problem of terrible management. That is what, regardless of your politics, is being slightly jostled by DOGE. Personally I have dealt with enough of our absurd government processes that I don’t think they can make anything much worse, and it cannot be less efficient.

[skybrian]

How do you know what the people involved did? Let’s not pretend speculation is fact.

[Natfan]

any process can be made less efficient, especially by firing those who are aware of how the system actually works...

[foltik]

They seemed to have replaced Mega Bloks with Legos, not skyscraper building materials.