[morty_s]

> it takes a special kind of new hire

Yeah, it does. I’ve had a good amount of experience doing this. You need to have tact and a kind of humbleness that can be difficult to maintain (tbh, it was for me at least). In my most recent experience with this, many of the bugs were unknown. During my ramp up, I read through nearly all of the code I’d be working on and got a good sense of what needed to be done, but the code changes were the easy part. The issue that required more work IMO was navigating the social waters as a new hire.

I found that you can use being a new hire to your advantage, eg “hey I was reading through this piece of code and I had a question about XYZ.”

Sometimes I’ve felt certain bugs are, for one reason or another, difficult to discuss. One thing I’ve done in cases like this is to refactor a bit of code in a way that makes a subtle bug painfully obvious; then in review, where the reviewers were the original author and reviewer, it’s easier for them to see the bug and say “this looks weird,” to which I respond “ah, yeah, this seems like a bug, I’ll fix this.”

The rough part of this is that you can get so good at being the fixer that this becomes all you (ie. maybe you’re not working on new features, etc) which can suck. The bright side is that you can strengthen the team—-everyone gets a chance to build/rebuild context and mental models, design docs can be updated, and the teams foundation gets stronger as a result. Of course, ymmv.

As a new hire, I find it important to build trust and relationships.

[cbushko]

This resonates with me so much as I was in this position 3 years ago.

My boss came to me and said "we want to move from AWS to GCP, using kubernetes and not have it be a mess like we have now"

I was new, senior, free from ownership. I chose to stay as a team of 1 so that I wasn't burdened with fires and meetings. It freed me up to put my head down and get the work done.

The grind was learning GCP, kubernetes, terraform, the platform/product, gathering requirements from several teams and mentoring people that would pop into the project as their time allowed.

I won't lie as learning new things was great. The project was partially greenfield as it was a platform migration and not a product rewrite. There were a lot of constraints & debt to deal with.

Maybe the new employee churn is one of the reasons that most startups get so much done... the new people are not burdened by ownership and being the subject matter experts. They are free to do things, good or bad. :)

[otabdeveloper4]

> we want to move from AWS to GCP

Out of the frying pan and into the fire.

[kakkan]

Why do you think GCP is worse than AWS? I've used GCP and found it quite easy to start with and the managed instances was a breeze to set up and deploy stuff. I've done it for only small businesses and they were happy with the results too and the easy to use console made things easier to navigate. Maybe the trouble with GCP comes with scale?

[otabdeveloper4]

I'm strictly against all "cloud" lock-in, but if you're going down that path despite all caveats, then using the #1 cloud provider makes more sense than a third-place competitor from a company that's best known for not having customer service.

[sangnoir]

Kubernetes is as anti-lock-in as you can get when it comes to cloud platforms. Which makes sense when you think it's the #3 competitor who mostly sponsors it - on the whole, lock-in benefits the market leader, so commoditization works to upstarts advantage.

[kelp]

I recently worked at a single digit billion dollar valuation SaaS provider and we had to run everything on AWS, GCP and Azure, to be where the customers were.

Total cloud spend was in 2 digit millions per year.

Many hundreds of k8s clusters, all using each providers k8s managed service.

My general high level take is this:

* AWS

Products mostly work quite well. Their UX is terrible and very inconsistent between products. But things mostly work, and their support is generally excellent, especially when you have a TAM (Technical Account Manager) who you can reach out to at any time. They will help you navigate any support issues, and ensure you get a quick resolution. But AWS stuff is generally very complex, overly complex in my opinion.

Also at scale API rate limits just kill you, and AWS continues to struggle to provide any real visibility into API rate limits. I'm told that large services internal to AWS have to work around this by building their own caching layer in front of any control plane APIs. Also the api limits are at the account boundary, so you have to start doing account sharding.

GCP:

Has better UX, more consistent experience. Much better visibility into limits and things like that. You still probably end up having to do project sharding at scale to work around limits. We continued to run into a lot of issues with GCPs load balancers blackholing traffic, and it was never really resolved. I think their support is generally worse than AWS. I've had substantially worse support experiences with GCP than AWS at 2 companies now. Mostly support not being particularly useful, and not resolving the issue. As a customer it feels like Google has struggled to adapt their culture to the needs of real enterprise customers. AWS has this much more figured out. But my GCP account teams have always been quite good.

Azure: The worst of all 3. Lots and lots of issues with their k8s service, low limits on the number of nodes you could deploy. k8s 1.18 supposedly fixed a lot. But their answer was always just that you have to upgrade. Their support engineers were frequently gave us actively bad advice that would have resulted in full customer outages if we'd followed it. The engineers on my team fortunately didn't listen and instead figured out how to fix issues themselves. Also frequently ran into various limits and you need to start thinking about subscription sharding.

Of the 3, I personally prefer using and interacting with GCP. But their support or is still certainly worse than AWS. That said, I always felt bad for our TAMs at AWS. It felt like a non-stop job where you had too be available at all hours to deal with unhappy customers. They usually gave us fantastic support, but I wonder at what personal cost.

[kakkan]

Thank you, appreciate the detailed answer.

[jen20]

> Why do you think GCP is worse than AWS?

Because the identity model is worse than IAM, and the quota system is insane. The global VPC networking being nicer doesn't make up for this, unfortunately.

[cbushko]

The fire is 1/3 of the cost of the frying pan.

[joshuamorton]

The best advice I was given (when I was a new hire, by my manager) was something along the lines of "You're smart, and you aren't burdened by tradition or seniority. If you see something that looks weird or confusing, ask about it! No one will hold it against you, and your point of view may keep us from committing to something more complex than it needs to be."

I've tried to keep that level of intentional naïveté.

[ErrantX]

intentional naivete is such a great way to put it, definitely going to use that going forward :)

[maroonblazer]

>The rough part of this is that you can get so good at being the fixer that this becomes all you (ie. maybe you’re not working on new features, etc) which can suck.

This is where a good manager makes a difference. They're also interested in your professional development. They should recognize the valuable work you're doing, have you train new(er) hires and move you into more senior positions, more interesting work, or both, whichever you prefer.