[beebmam]

Disagree. A senior engineer should make evidence based decisions, as should any engineer. If there's missing evidence for a conclusion, then a senior engineer should be able to design experiments which gather more evidence to derive a conclusion.

Senior engineers should not be con-men.

[CapitalistCartr]

That sounds great, but hard decisions are hard because there isn't time, money, other resources to have all the facts, yet a decision must be made. There might not even be any way to know for sure without taking one path. Experience is all we have to guide us. Progress is full of such decisions, made in a gray fog. Fear of being wrong is paralyzing.

[galaxyLogic]

Exactly. My favorite metaphor for SW-development is playing chess. When you make a move, a decision, new choices, new decisions to make, open up and other ones may no longer be practical.

What you decide and do now, affects what you can and must decide next.

Except unlike in chess we don't really have the exact rules of the game anywhere. Still we must make choices which affect what moves can be chosen later.

It is also like building a house, once you decide how to build the ground-floor and what it stands on, that greatly affects how the floors above that can be built (not too heavy, not too light etc.).

It is often difficult to understand all the consequences of a design decision, what can be done and chosen after that, until you actually implement that design.

[aYsY4dDQ2NrcNzA]

Path dependence.

[galaxyLogic]

I googled path-dependency https://www.investopedia.com/terms/p/path-dependency.asp#:~:....

It happens all the time of course. But I wonder if it is a bit of a trivial concept. Of course what was done before will affect what is the optimal choice now.

But I agree it is good to keep it in mind -- when you are making the current decisions. If I decide this way now, what is the cost of my available options in the future? Pretty much the way chess-players evaluate their possible moves.

[jldugger]

> That sounds great, but hard decisions are hard because there isn't time, money, other resources to have all the facts, yet a decision must be made.

While I am sympathetic to the concept of reasoning under uncertainty, the way you've framed it is hardly engineering anymore.

Design projects come with a spectrum of data requirements, and each has its own cost function for improved accuracy. For example, knowing exactly how long the bridge you need to build matters greatly, and the cost of surveying should be pretty low. In contrast, measuring the soil at the site you build on top of has a variety[1] of options with varying costs. You can't ever be 100 percent certain that your sampling was good enough, but the samples more you have the more confident you can be. Depending on the project and design, you may not need the most expensive and most accurate option, but you probably don't want to just rely on priors built through "experience."

Translating this to the world of software, we have a variety of data collection strategies. A/B tests, canaries, software instrumentation, tracing, offsite monitoring, f-scores, beta tests, market surveys, and more. The best thing you can do as an engineer making an uninformed choice is to design the system to collect the data you are missing and to make it easy to change your design later. Amazon calls these "two way doors" versus "one way doors"; if your decision was wrong you can go back and make a different choice. As an example, if you are designing a caching system, spending a bit of time to generalize the API will allow you to swap between policies as you collect data about cache hit rates. If you don't, the cost of changing your mind goes up as more things depend on the nuances of your API.

Rather valorize making uninformed decisions, I'd prefer to valorize flexibility and continuous data collection. Seen this way, the old adage "There's never enough time to do it right, but always plenty of time to do it over" is a coherent philosophy.

[1]: https://en.wikipedia.org/wiki/Geotechnical_investigation#Soi...

[f1shy]

If you have to make decision, and lack information, time and money to even try something to understand the best decision, you are having problems bigger than technical ones; and even the best engineer in the world will probably not do better than a dice

[MAGZine]

> A senior engineer should make evidence based decisions, as should any engineer.

... sooo there is no distinction between a senior engineer and a non-senior engineer? you've really said nothing in this comment.

senior engineers have seen enough things to know what the right solution is in many cases, without having to take a bunch of time to collect evidence. that's what makes them senior, not just 'good'.

[lowercased]

> seen enough things to know what the right solution is in many cases

When dealing with uncertainty (nearly always) it's generally easier to identify and rule out what 'wrong' solutions are. There are often multiple obviously 'wrong' options (no, we should not keep a user's password in plaintext, even if the goal is to make it easy to recover a lost password), but picking a 'right' one... from a whittled down list, it may often come down to familiarity or convention rather than an arbitrary "this is the only 100% correct solution".

[f1shy]

Then the problem is not missing information, as stated. There is enough information, if you have enough experience.

[beebmam]

Disagree, profoundly.

[Nevermark]

An important series of skills would be:

1. Identify every project dimension with risks. The big one: project -> use -> user -> reward -> costumer fit? (Where user and customer may, or may not be the same.)

Price cost economics? Technical complexity? Technical experience shortfalls? Project resource availability? Solution resource efficiency? Safety? Third party dependencies?

2. Identify each significant risk in each risk dimension.

3. Identify the simplest question (or two) whose answer will reduce or eliminate each risk.

4. If a risk can’t be mitigated by answering a single question (or two), maybe it is really a combination of risks, or risk dimensions. If so break them out, repeat.

4. Craft the simplest research task to resolve each risk question.

—

This sequence is a special case of focusing energy on upstream tasks, before investing in downstream tasks.

Or as I like to say: move slow to move fast.

—

A meta risk is loops in risk/work dependencies.

Classic example: Stakeholder wants to see something working before you resolve risks of getting something working.

Identify the simplest, sparsest form of a loop is one way forward.

Alternatively, break the loop. Identify alternate ways to satisfy the stakeholder. Find a way to remove dependency on the stake holder (constructively).

[stinkbutt]

A senior engineer should work in an organization that actually enables this type of engineering

[delusional]

I can kind of agree with your sentiment. We should run experiments, regardless of our skill level. We should, in fact, regard anything we do as an experiment. Very few decisions in the real world are irreversible. Doing experiments doesn't mean there's no skill to it. The Senior engineer will know what hypothesis is most likely to prove true, and will start with a system design that will work out well assuming that hypothesis to be true.

This is what the previous commenter referred to as "making a decision without all the evidence". If the senior engineer turns out to be right, and the hypothesis proves true, the experiment becomes the system. If the hypothesis turns out false, the system might still be workable, otherwise it's refactored into some new experiment.

The senior engineer is better at making that initial guess as the correctness of a hypothesis. Just like a senior scientist.

[paulcole]

There’s always going to be missing evidence. Almost all business decisions can be made without 100% certainty. Make your conclusion with what you have available and find out if that’s good enough. Don’t dawdle trying to get more info that’s probably not needed.

[kuchenbecker]

Philosophically I agree, but in my experience the company isn't willing to pay for experiments except in Production due to scaling issues and quality of synthetic data.

So you need to instead have a series of features that are architecturally solid, have KPIs to measure success, and be willing to throw out working production code that doesn't move metrics enough to warrant their complexity.

[bsder]

Sometimes a decision needs to be made today. That's just life.

Engineering is all about what you do when you don't have perfect information.

This is why real engineers have things called "safety margins".