Hacker News new | ask | show | jobs
by m0ther 2261 days ago
I am a senior software architect. My job is to balance performance against complexity.

If my system is slow, it's my fault.

If debugging or expanding the system is too difficult, it's my fault.

If someone wants to know how the system or business works in depth, I am the one that they should come to.

I spend the majority of my time chasing down, enforcing, and simplifying the universal theory of our business (the core of our software solution).

The universal theory of our business is a living collection of concepts, designed to accurately model non-virtual concerns in virtual space. If there are too many edge cases, it is a sign that the universal theory is inaccurate, or not robust enough in some areas. If there are too many bugs, it is a sign that the universal theory has not been communicated or enforced well enough, is inaccurate, or is too complex in some areas. As our business grows, or our understanding of the business expands, the universal theory will change; at times dramatically. Malleability (the ability to adapt our software to these changes effectively and efficiently) is one of my top two concerns; the other is latency (how long it takes for any one request to get a complete response).

The theory shrinks and becomes better documented as it evolves; the goal is to move from describing behaviors as correlation to describing them as accurate causation. To fill in the blanks.

I should mention here that this does not mean every line of code in my projects is easy to understand. Writing a fast system of high complexity requires at least some components that are written exclusively for the computer's benefit (that is, highly optimized and inherently difficult to read). These components should be written with clear documentation, clearly defined public members, written discussions of why it works the way it does (and common ways to accidentally break it), redundant ownership, and regular auditing to ensure code rot is avoided.

I have yet to meet another architect that sees their job the way I see mine.

If the system you're working on defies complete understanding, you can probably blame your architect.

5 comments

Thank you for a beautiful description of your philosophy, and your philosophy itself. I wholeheartedly agree with you, and see things the same way; though I haven't seen it expressed so clearly before.

I'm curious – and a little saddened – that you haven't met anyone else thinking this way?

Anecdotally, there are maybe 10 out of 200 engineers at my current company who demonstrably take this "Quality of Craft" part seriously and explicitly: they think in a similar way to you.

It's a rare set of skills: empathy for future humans, understanding how the universal theory is likely to evolve, caring enough about it to enforce it consistently, and effectively actuating on all of these thoughts and emotions. Those add up to somebody special, I suppose.

Anyways, I think I'll link to this comment many times in the future; thank you again.

I am not aware of a standard (and in depth) definition of what a software architect is; other than it being the next step after team lead, and the last step before pure management (unless your company offers a research track - which nearly no one does). The interpretations I've run into are pretty varied. My way is certainly not the only way, and my way is an amalgamation of aspects of others that I admired.

For reference, I'm 36, have been programming professionally for 20 years, and have been an architect for 10. I seek out difficult projects.

Here are some (likely idealized) descriptions of my favorite past architects:

"Never say no"

My first architect took immense pride that at 40 something, he had never had to say no to a product manager. We were going to build whatever the business wanted, with no push-back (only what he called lateral guidance, or "yes, and"). The business would arrive with specifications, and he'd treat them as if they were rough drafts; "OK, let's get to work". He'd disappear into meeting with the business, and emerge days later with a nearly completely rewritten specification (which he had written himself and was usually more feature rich than the original), and everyone was happy. He explained to me after a few years that most bugs and development problems come from the blind-spot the business has for how software works, and the blind-spot development has for the business. A specification with no blind-spots is much easier to turn into software. Every project I worked with him on went smooth, and was delivered on time. I strait up stole and expanded his technique as the foundation of my architecture philosophy.

"Make me one"

I worked with an architect that insisted everything be built in-house (no unnecessary external dependencies). He'd look at the features of some other framework or tool, copy the bits he liked out of their documentation (as if they were requirements), and turn to us and say "now we're going to build our own". I loved this guy, he pushed me so far outside of my comfort zone that every day was a frantic adventure. "Write me a sketch; you have 4 hours". He didn't like whiteboards, he liked code and rapid iteration. He was brutally honest and ripped my code apart at least twice a day. We'd play code tennis for a couple of days, then a polish phase before QA. I was in hyper-focus for 8 hours every work day, and the practice at rapidly building (and rebuilding and rebuilding) all sorts of difficult components (that all made it to production) made me fearless. I lost that job to the 2008 bubble burst, but the year I spent there easily advanced my skill set by 5 years.

"If it's not fast it's useless."

I worked with an architect who came from systems programming. He had an incredible resume. He spent a lot of time refactoring for performance. If he liked you, he'd explain what he did to your code and why; if not, that code became his now. He spent a lot of his time in instrumentation, testing this solution against that solution for execution time and memory usage, merging the best bits, and testing against a different approach. His favorite phrase was "prove it" (and I spent a lot of down time trying to prove things to him through examples and instrumentation). I learned a ton in an effort not to disappoint him. It was no longer enough to know how to do something; I had to know many ways to do it, and which performed better in what scenario. "What, as an individual programmer, do you bring to the table if not performance? If your solution doesn't perform, you let everyone else down." He dramatically and permanently changed the way I code and the way I design applications.

"If it's not predictable I hate it."

I worked with an architect who's main focus was the application as it ran in production. He spent a significant amount of his time combing through debug logs and recreating log messages locally to understand what's going on, tracking down run-time issues that customers called in with, building instrumentation and internal tools, looking at characteristics of the application as it ran with network operations, and treating the production application as a living thing. I learned a ton from him; he changed the way I look at applications.

I've also worked with a number of architects who I did not like, and who I learned nothing from. Among them:

- The guy that misinterpreted the book Clean Code and turned the code-base into a ridiculous mess

- The guy who made decisions like we were Google, when we most certainly were not

- The "lead by Lint" guy

- The design by committee guy

- The "non technical" architect

- The "I rebuild most of the application last night" guy who just made it worse

- The "I read it in a blog so it must be both true and universal" guy

I love it! I wish I could have some of those difficult projects. I've sadly been wearing down my curiosity and idealism in favor of pragmatism and economy for the last few years.

How do you find work where this quality of code is valued? I can produce an entire machine in a couple hundred hours of work that will make my client money for a decade until the product line is cancelled and the machine and the time and money I put into it is depreciated. In that couple hundred hours, there's some time for problem modeling and architecture considerations at the start, and usually at least one or two really hard problems where nothing off-the-shelf will work and you have to "make yourself one" that might take a few days each, but the vast majority of the time is just taking in business logic, translating it to code, and testing/documenting.

I treasure the times I get to solve an interesting problem or can develop a system that's beautiful and accurately models the system required. But the sad truth that I'm coming to understand is that in a non-software company you can get a lot done for not a lot of money by duct-taping together purchased components and copy-pasting a simple, versatile component a couple times. I believe there are businesses where this pragmatic approach won't work - there's not much done in the space yet, the project will live long enough and evolve so it needs to support rework, and you're somewhere between Google and a one-hit wonder. But industrial automation is not one of those businesses.

This was fantastically entertaining and educational. Sign me up for the long-form version!
> The universal theory of our business is a living collection of concepts, designed to accurately model non-virtual concerns in virtual space.

Isn't this ontology?

https://en.wikipedia.org/wiki/Ontology

> ontology often deals with questions concerning what entities exist or may be said to exist and how such entities may be grouped, related within a hierarchy, and subdivided according to similarities and differences.

I've always thought of software as a reflection of its author's understanding of reality. The more the we learn, the better we can express that reality in the form of software.

Re. ontology: If that's not it, it's close.

Business logic is certainly a reflection of a programmer's understanding of the business, as well as their skill in applying object oriented concepts (or whatever) and occam's razor to express it in a manner that doesn't balloon in complexity and isn't cognitively wasteful. There's an art component, and a science component; and if they're both right it will make a solid foundation for everything else, and for communication with the business.

I like what you've written here. If you write stuff elsewhere, please put a pointer in your HN profile.

Judging by your influences as an architect, I think you would enjoy these books:

- Ronald Ross, Principles of the Business Rule Approach.

- David B. Black, Wartime Software. If you like this, you'd probably like his other books as well. They each cover a different facet of software development, including QA and "project management" (which Black refers to as a disease).

You might also enjoy the essay by David Black in which he proposes "Occamlity" as the metric of software "goodness" (https://www.blackliszt.com/2020/03/william-occam-inventor-me...):

> I propose that a piece of software can be measured by its “Occamality.” The more “Occamal” it is, the better it is. And I propose that Occamality is strictly correlated with the extent to which there is no redundancy of any kind in a program...

I didnt believe it could be said in few sentences the gist about the things I do and why my approach is unique. Thank you.
I'm glad I'm not the only one :D
Unfortunately very little effort is done to question and push back on the requirements that brought all that complexity and performance problems in the first place. If you can never say 'no', if there is never time to clean up, balancing performance against complexity gets harder and harder over time.
I like this take on software arch. essence.