Hacker News new | ask | show | jobs
by cellularmitosis 2613 days ago
Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency story and cut the latency by 75%."

lol yeah that guy should totally be fired.

7 comments

Another developer: "...our stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. I just bypassed the ORM and wrote a slightly more complex SQL query, so that there is no more N+1 query problem in the backend. The latency of this end point was cut by several orders of magnitude, and our database is not overloaded by parallel inefficient queries, run by clients with 'great concurrency stories'."
Or, "here's a completely supported not-well-enough-known ORM feature that does that, that I learned about when it looked like I wasn't doing any work".
From a technical perspective, bypassing the ORM is a great solution (and if you can get the number of queries down to one, there’s no longer any need for concurrency).

Unfortunately, politically, the kind of shop which fires a dev for choosing another platform is unlikely to approve a PR which bypasses the ORM (maybe also a fireable offense!)

If there are no other people who can maintain that stack, maybe (firing is too extreme, I’d just ban the Clojure code in the production)

Writing initial version of the code is only a small part of it. Maintaining and enhancing it is much harder.

What is the company going to do if the new endpoint went down and the author is on vacation?

> just ban the Clojure code in the production

A dev ran up against the limitations of one platform and solved the problem by choosing a platform which didn't have those limitations. And management responded by banning the new platform.

How long do you think that dev is going to stick around in an environment like that?

Ultimately this comes down to what sort of environment you want to foster. One approach leads to an environment where devs have lots of freedom, and that creates certain problems. The other approach eventually leads to an environment where even the choice of editor / IDE is dictated, and that creates a different set of problems.

I'd favor attracting a bunch of smart folks and give them as much freedom as possible. I'd much rather deal with the kind of problems which that sort of environment creates.

I am not sure why you think it is "management responded" -- it is all about other people on the team.

Right now, I am responsible for maintaining one of our internal services. If someone tries to rewrite part of it in Clojure without extensive internal discussion first, I'll do all I can to stop it (and I am not "management" at all). And most people on the team will agree with me.

If the original dev would not want to stick around after that, good for them. If they do not care about teammates at all, they would likely be happier working alone anyway.

After all, single-handedly moving to a new technology and then forcing everyone else on the team to support it is impolite and annoying. Clojure is nice, but I don't want to be forced to learn it because the server went down on my shift.

(this is different if the service does not need to be up all the time, or the company is very small. But if you are talking about API endpoint being overloaded, the chances are, neither of this is true)

“If someone tries to rewrite part of it in Clojure without extensive internal discussion first”

“single-handedly moving to a new technology and then forcing everyone else on the team to support it is impolite and annoying”

The dev you’re describing has indeed made some bad choices and I wouldn’t want to work with that guy either.

What I had in mind was more along the lines of “check out this thing I made, what do you think?”

Totally ok responses would include: “let’s just bypass the ORM here instead”, “let’s divert 5% of traffic to it and see what happens”, “this stays a prototype until you reach a bus factor of 2”, etc.

A lot of these objections depend on context, though. The culture at large organisations taking on large and/or long-running projects is necessarily different to the culture at small organisations that can't afford the same overheads and wouldn't enjoy the advantages of greater redundancy or interchangeability anyway.
The problem here is not understanding your database not about making a system in an unsupported and probably undeployable thing to prove how clever you are...
> "to prove how clever you are"

I see this a lot and I just don't get it. Is a mechanic who uses air tools trying to "prove how clever he is"?

Knowing more than one tool allows you to choose the right tool for the job. Assuming arrogance and a motive of "proving cleverness" sounds like a great way to pick a fight.

this is not a valid comparison.

a mechanics' tools would be IDE, build tools etc- I'm not interested how every dev uses personal level tools.

But just writing a program with a new dev stack without thinking of the possible consequences is just plain unprofessional.

Air tools are fine -- after the car left mechanic's garage, I cannot tell if air, electric or manual tools are used.

But if I give my car to mechanic for carb cleaning and tune-up, and get it back with fuel injection system and explanation "it is better anyway", I would be very unhappy.

There are no parallels in the real world to how programmers chose and use languages, frameworks and tools.
He should get fired for saying "concurrency story".
> lol yeah that guy should totally be fired.

Yeah, introducing a new ecosystem to mask bad DB design or bad queries because she is lacking basic SQL and relational foundations is how I envision the typical 10x programmer.

I hope you are being sarcastic.
Maybe not. But if the alternative was spend one tenth of the time to cut the redundant sql queries down a bit and cut latency 50% - leaving a codebase that anyone on the team could further develop and debug... Then "that guy" might stil be in the wrong...
Rather than making thousands of requests in parallel, wouldn't reducing the number of queries be the more obvious choice.
Both would be ideal.