I share the opinion of the parent comment, pretty much exactly.
Some of my comrades call this a layer-8 problem. (hint: there is only 7 technology levels, 8 is the human one).
One story in particular my firm was hired to come into a startup that had failed to scale, both technology wise, and personnel wise. They had hired up to 25 engineers, hoping to fix their stability issues, and almost going bankrupt doing it. I am at no liberty to say which company this is publicly unfortunately.
It was stressful, but you can't take things personal. You have to put aside feelings, to get useful data out of these people to make the platform work. You can't possibly care about someone's ethics, approach, and often even coding style on these missions. The goal is to try and not piss off as many people as you and, and go ahead not worry if you do have to piss someone off to get the job done, and not lose sleep at night if someone does not get along with you.
The end result was that they had to shed, through various reasons the great majority of their internal team. We stabilized things and bought them lots of time, while they slowly brought in new senior tech management and rebuilt their internal team from the ground up.
It was a multi year affair, and involved working on over 12 codebases and consolidating applications from 3 different hosting providers into AWS.
They got very close to running the ship into the ground. If you think staying up late and pulling the occasional all-nighter is unhealthy, or having a job that has extremely high expectations and leave you no time for a personal life is bad, this is about one hundred times worse. Since my team is small and highly skilled we can get a lot done in a short amount of time, and occasionally we are terse even with each other. But the amount of sleep deprivation that comes with a task of gargantuan size, where your dealing with hundreds and thousands of requests (sometimes per second), the company is going for broke, and a single code change can improve monetary situations in instantaneous, tractable ways, it gets super intense.
You have the CEO, the COO, the CFO all breathing down your neck. One day they see light at the end of the tunnel from fixing the currently broken problem, and then the next day, you discover another codebase lost in git that powers commerce for android, is starting to experience issue, and you don't even know where to find credentials to log into its production systems since it was setup by some employee in the middle of the night a year ago that is no longer there.
I can go on and on and on. But honestly, please don't read this as a recommendation. Being a digital mercenary is fun in your 20s, but it got old quick. Of course I still do it, but I have a much healthier way of saying NO, more often now. You can make a very lucrative living doing this sort of work, but its hard to earn the reputation to get these clients, and you may end up forgetting what your family looks like by the end of a 2 year job that is 365/24/7.
The story did have a happy ending though. The company does well today. Extremely well. A new era of management has come in over the last year, and I think everyone learned a lot. They are a household brand and I hear about them in the news it seems like monthly.
>you don't even know where to find credentials to log into its production systems since it was setup by some employee in the middle of the night a year ago that is no longer there.
Recently I was searching for some report generating code that my team had been voluntold to maintain. I spent a day trying to find the code that generated and uploaded the report. Once I finally was able to get in contact with the last person who had worked on it I found that a developer in India was manually running and uploading the report every week. Why? Because job security.
"Layer 8 Error" is one of my favourites, classier than PEBKAC or ID-10-T. It's based off the OSI 7-layer networking model. Next layer up from the application? Must be the user...
>you don't even know where to find credentials to log into its production systems since it was setup by some employee in the middle of the night a year ago that is no longer there.
I've been the 'some employee'. I setup the prod systems of a former employer's client. While I did hand over the credentials, for the first few months after leaving I expected to get a call from my former employer asking for them.
I think there's only one that I'm comfortable talking about since it was long ago and the people involved have all moved on from their parent companies.
Louis Vuitton was relaunching their public-facing website, which was tightly integrated with their back-end systems - inventory, manufacturing, etc. IOW, it wasn't just a regular website.
They hired a consulting company here in Europe who sold them a solution that was the epitome of using the wrong tool for the job - it was a non-relational database solution for what was obviously a pretty traditional relational db application. (except the product content - images, movies, etc, had to be tied to their back-end db systems)
Louis Vuitton rented out the Louvre for the launch party, hosted by the CEO. But no one took it upon themselves to let the CEO know that the rewrite would not be done in time for the launch. Not even close. The launch party happened but the new site wasn't ready. There was nothing to present at the launch.
The CTO was fired of course. We were hired to help bring the project back on track along with a couple of other consulting companies - it was about 5 senior developers and a few dozen junior developers.
However, the work was being done in France where the norm is to not work under pressure like what we had to do. And the senior consultants where all american, swiss, and german, and only one very good and amazing french developer. I spent most of my time working on politics and very little time developing. We managed to get some of the work away from the managing French company to be done by Swiss and German companies. And then I was transferred to another project with another desperate client.
I was in Hyderabad a couple of years later and met the fired CTO at a party - he was starting a consulting company in India to provide consultants to Europe. Nice guy but meeting him made me understand why what happened happened.
Some of my comrades call this a layer-8 problem. (hint: there is only 7 technology levels, 8 is the human one).
One story in particular my firm was hired to come into a startup that had failed to scale, both technology wise, and personnel wise. They had hired up to 25 engineers, hoping to fix their stability issues, and almost going bankrupt doing it. I am at no liberty to say which company this is publicly unfortunately.
It was stressful, but you can't take things personal. You have to put aside feelings, to get useful data out of these people to make the platform work. You can't possibly care about someone's ethics, approach, and often even coding style on these missions. The goal is to try and not piss off as many people as you and, and go ahead not worry if you do have to piss someone off to get the job done, and not lose sleep at night if someone does not get along with you.
The end result was that they had to shed, through various reasons the great majority of their internal team. We stabilized things and bought them lots of time, while they slowly brought in new senior tech management and rebuilt their internal team from the ground up.
It was a multi year affair, and involved working on over 12 codebases and consolidating applications from 3 different hosting providers into AWS.
They got very close to running the ship into the ground. If you think staying up late and pulling the occasional all-nighter is unhealthy, or having a job that has extremely high expectations and leave you no time for a personal life is bad, this is about one hundred times worse. Since my team is small and highly skilled we can get a lot done in a short amount of time, and occasionally we are terse even with each other. But the amount of sleep deprivation that comes with a task of gargantuan size, where your dealing with hundreds and thousands of requests (sometimes per second), the company is going for broke, and a single code change can improve monetary situations in instantaneous, tractable ways, it gets super intense.
You have the CEO, the COO, the CFO all breathing down your neck. One day they see light at the end of the tunnel from fixing the currently broken problem, and then the next day, you discover another codebase lost in git that powers commerce for android, is starting to experience issue, and you don't even know where to find credentials to log into its production systems since it was setup by some employee in the middle of the night a year ago that is no longer there.
I can go on and on and on. But honestly, please don't read this as a recommendation. Being a digital mercenary is fun in your 20s, but it got old quick. Of course I still do it, but I have a much healthier way of saying NO, more often now. You can make a very lucrative living doing this sort of work, but its hard to earn the reputation to get these clients, and you may end up forgetting what your family looks like by the end of a 2 year job that is 365/24/7.
The story did have a happy ending though. The company does well today. Extremely well. A new era of management has come in over the last year, and I think everyone learned a lot. They are a household brand and I hear about them in the news it seems like monthly.
Cheers