[ianbutler]

Idk man my system design game is better than its ever been because I put in the effort to use these tools and recognize they can't do software design better than I can and because I've increased the scope of what I'm building I often have to think more deeply about the problem up front. A typical speccing sessions lasts a few hours for me on big work before I have AI start writing that work where I'm just going back and forth on what I want, points of consideration for performance, usability, structure etc pushing back on where AI (always) chooses the most naive way it wants to do something.

Every time I see an anecdote like this, A it reaffirms my belief that FAANG devs are fairly mediocre on the whole (not saying this is you, obviously there are good FAANG devs) and B it reaffirms my belief that the developers who kind of give up their thinking like this are really using the tool wrong or didn't really care about the work before AI either so its now just a quick means to an end.

[jcgrillo]

> Every time I see an anecdote like this, A it reaffirms my belief that FAANG devs are fairly mediocre on the whole

I think another (partially causal?) problem is how they're managed. The whole perf circus is just ridiculous, especially the stuff recently reported about facebook. But they're all more or less like that. Steeped in that cocktail of incentives, who even knows what might happen to an otherwise excellent engineer.

But also just numerically, they can't be much above average, on average, because there are so many.

[iLoveOncall]

> But also just numerically, they can't be much above average, on average, because there are so many.

There are as many CS grads every 2 years in the US alone as total software engineers employed by FAANGs...

[jcgrillo]

I stand corrected! That's very interesting.

[Lwerewolf]

+100 on this. In addition, if you don't outsource your thinking and you're willing to go through all this, you absolutely don't need the top tier models.

[iLoveOncall]

I strongly believe that you cannot evaluate how good a system design is if you don't implement it by hand.

LLMs will implement what you ask them to, even if it is the wrong approach. They can be lazy and take shortcuts all the time, but they do not feel PAIN (obviously they don't feel anything and aren't lazy, I'm just personifying them but you get the point). Only when you implement by hand can you feel if the implementation of your design is painful or not, and only this signal can tell you if your design if truly good or not.

I do think LLMs are useful for design work, they are good at asking clarifications and probing questions which actually do push you to approach problems differently, but leaving implementation of designs to LLM is a recipe for disaster, and judging your own design skills when you're not implementing such designs is seriously laughable. And to be clear, it already was before LLMs, when "software architects" were just designing and then had peons implement for them.

LLMs are enabling a whole new level of bad code that is best describe by the following Jurassic Park quote: “Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.”.

> A it reaffirms my belief that FAANG devs are fairly mediocre on the whole

Off-topic but having worked in other companies as well, I can guarantee you that this is not the case. The skill of engineers in FAANGs and other "top tier" companies is much higher than average.

[ianbutler]

> I strongly believe that you cannot evaluate how good a system design is if you don't implement it by hand.

Fwiw we both agree that LLMs should not design systems. I do the design, but otherwise I don't get how this is true, the success of a design is indicated by long term success in the system it built. You can measure this against success in the task it was deployed for via performance metrics for one. And then from a developer standpoint how easy it was to maintain later on. Success of a system is a measurement over time, but it's not some quality that can only be measured by those who built it.

> Off-topic but having worked in other companies as well, I can guarantee you that this is not the case. The skill of engineers in FAANGs and other "top tier" companies is much higher than average.

I have first hand knowledge of this so I agree to disagree. Being surrounded by google, aws, and meta folks my understanding is the best people leave faang when they get the itch to do something better with their time.

[iLoveOncall]

> my understanding is the best people leave faang when they get the itch to do something better with their time.

Your understanding is wrong. The extreme majority of people who leave FAANGs don't leave willingly, and are not the best, unless it's to go to even better companies like Jane Street and the like, but those are rare.

Turns out for most people, there's not much better to do during your 9 to 5 than solving problems for half a million a year.

[ianbutler]

> Turns out for most people, there's not much better to do during your 9 to 5 than solving problems for half a million a year.

Lol. I don't know I was talking with a guy yesterday who left a FAANG clearing 1.4 million / yr who's now running his own successful startup. My sample is successful founders backed by top tier VCs or exited founders who have done much better than FAANG. If you can't do more you stick around the FAANG, those who can go and do it.

[skjfjnflw]

You're talking about what happens to engineers who decide they want to be small business owners rather than engineers. That's orthogonal to this thread about where engineers with the best technical skills end up.

[ianbutler]

Most of the best engineers I know leave faang because the work at faang sucks. It is not orthogonal. The majority of faang work is not highly impactful work like trivially because the amount of work outstrips the amount of interesting work by an easy oom.

FAANG doesn't straightforwardly retain the best technical talent, and the median task there is routine.

[jcgrillo]

> Only when you implement by hand can you feel if the implementation of your design is painful or not, and only this signal can tell you if your design if truly good or not.

I think this touches obliquely on a point I keep coming back to, that one of the most important things a codebase does is to communicate ideas about how a process should work. Yes, it also produces some binary that runs on a bunch of servers or whatever, but that's a really temporary, ephemeral artifact. The lasting thing is the idea. Making your ideas (expressed in code) easy to understand, easy to work with, and easy to evolve in time is the art of software engineering. I 100% agree, from my own experimentation with LLMs, glancing at something a model has produced and checking that it has some test coverage isn't enough to know whether it's well-engineered. You'll only find out later when you try to work with the code.