[righttoolforjob]

Even though there isn't a formal theory, there definitely is a big difference between a person capable of doing good system design and one who isn't. I don't think all people are capable of becoming a systems designer or architect. It is not always required for projects, but when it is necessary it can make a great difference, putting a project on the right track from the beginning and keeping it on track, making sure it all becomes something good in the end.

I've done it enough to know that I couldn't do it while keeping my IC-type role as well. For me, personally, I approach systems design in a sort of breadth first search for a solution to the entire problem space, whilst deep-diving into particular areas where I am less certain. For other parts my experience lets me brush over details quite quickly.

However, that is quite different from an IC-type role, where you'll typically run into a multitude of practical engineering issues that can be frustrating and take a lot of time. Tool issues, cloud issues, bug hunting, smoking out every little detail, write unit tests, review others code, etc. That would completely throw your mental cycles in the wrong bucket.

I don't think you should do system design without having plenty of experience from IC-type roles though. You have to have understood so many different aspects; capabilities of the people at hand, the organization at large, tools, frameworks, technologies, etc, and be very, very willing to communicate the picture over and over again and take in feedback as the project progresses. It's technical leadership at one of the hardest levels.

But realise at the same time that even software development itself doesn't have anything resembling a universal theory, common processes or frameworks. It's all changing, being reinvented, and rediscovered on a continuous basis. It is equally true that many things even in medicine or construction aren't based on any solid science.

[kilgnad]

>Even though there isn't a formal theory, there definitely is a big difference between a person capable of doing good system design and one who isn't.

Yeah? Prove it. You can't. That's the big problem here. The only thing you can give me at best is some vague metric on some anecdotal experience. We don't even have data on this. Replace "system designers" with ICs who have the same breadth of IC experience who produces the better design? What does "better" even mean?

My personal opinion on Systems design is that it's pretty easy if you got the "IC" part down. It is largely just arrows and boxes.

>I've done it enough to know that I couldn't do it while keeping my IC-type role as well. For me, personally, I approach systems design in a sort of breadth first search for a solution to the entire problem space, whilst deep-diving into particular areas where I am less certain. For other parts my experience lets me brush over details quite quickly.

I think of system design as something that is tediously hard. It's not something that requires a massive amount of skill. But it does take a lot of time to come up with a design. It's like building the Eiffel tower out of toothpicks. Anyone with the relevant domain knowledge (as opposed to design knowledge) can do it. So yeah it does make sense you can't be an IC at the same time.

>But realise at the same time that even software development itself doesn't have anything resembling a universal theory, common processes or frameworks. It's all changing, being reinvented, and rediscovered on a continuous basis.

This is the whole point. Anything referred to as "design" is largely an artistic endeavor. Arguably much of these frameworks have been changing in a manner that cannot be characterized as "improvement". Just endless horizontal progress and endless genetic drift because we can't know if one design is better than the other.

This is the same with system designers. The skill level is horizontally stacked because we literally can't prove shit.

>It is equally true that many things even in medicine or construction aren't based on any solid science.

Medicine is a highly quantitative endeavor. All medicines go through rigorous quantitative verification for efficacy. The same is definitely not done with system designers.

[righttoolforjob]

I can't argue with your points about not having evidence, but I refuse to brush it off as "merely anecdotal" as well.

> I think of system design as something that is tediously hard. It's not something that requires a massive amount of skill. But it does take a lot of time to come up with a design.

Well, zoom out a bit, and imagine that this tedious and hard work where you are documenting and explaining things with arrows and boxes actually takes up most of your time. That's the birth of the explicit role.

I am also sure you can imagine people you have worked with that you would never want to have above yourself in such a role when you are in an IC-role, because they for example always come up with crazy ideas that won't work, or don't come up with ideas at all, or just brush over everything with simple arrows and boxes and assume someone else will figure the hard details of the overall picture out, and in fact in doing so will find no use for the arrows and boxes they got handed other than being some picassoesque requirements input from which you have to make real investigations and conclusions.

That's the people that aren't mature enough for such a role and I claimed many never will be.

But yes, I resonate with a lot of what you are saying.

[kilgnad]

I still disagree. It's just not that hard. I can explain by comparing it to art.

Artistry is hard and artistry is basically the same thing as design but harder.

Let's take something like say painting. Painting requires a lot of skill. Why? Two reasons. The sheer amount of ways to compose paint into a painting is astronomical. Much much much More then the amount of atoms in the universe. The amount of these paintings that would be considered "art" is also astronomical in number but the ratio of art to all possible paintings is minuscule in number and can basically be rounded to zero. This ratio and the actual huge numbers plugged into the ratio illustrate how hard artistry is.

How about building art or designs with legos? Suddenly it's easier. Why? Because the number of ways legos can be composed in a limited space is much smaller then the ways you can compose paint on a canvas. This is because lego Blocks have specific rules. Two lego blocks have a countable number of ways they can be composed, two brush strokes can be positioned with enough variation that composition is more or less infinite. This is why designing something in legos is EASIER than painting. In fact this is the part where the word "artistry" starts to transition to "design". In legos it can be said you are "designing" a structure. Basically when the skill involved with the creative endeavor becomes significantly easier we tend to transition from artistry to the word "design". It's not a hard rule but definitely a generality that exists.

I look at these constructs made by a "lego artist" and I know I can ALSO make those big constructs if I had the will and time to do it. It's a feat of enduring tedium and not much skill. But the mona lisa in oil paints? you need skill to render that.

Do you see where I'm going here? What's system design if not putting together lego like primitives? It's trivial. The only thing you need here is knowledge about how the primitives work and how they compose. That's really the only challenge.

Heck you can even write a program that has all possible "boxes" as primitives with the right composition rules and just brute force evolve a design through random compositions. Each design has what? at most 40 primitives in a typical design? Maybe 200 total primitives? Let's make it 10,000 total system design primitives just to be excessively generous. Even at that number, system design is easy enough that it's a candidate for genetic programming.

Do you think such a thing can be accomplished with pixels? Randomly generate a square of 1000x1000 color pixels until you get something that looks legit? Running it at 5000 generations per second you probably won't get anything legit before the sun goes super nova. You'll need to switch out of random walk to machine learning to get anything artistic.

[righttoolforjob]

> The only thing you need here is knowledge about how the primitives work and how they compose. That's really the only challenge.

> I can explain by comparing it to art. Artistry is hard and artistry is basically the same thing as design but harder.

So you agree it is kind of hard, requires a lot of knowledge about components and how they can best be composed.

Q.E.D.