How do you learn that solution curve and their tradeoffs? Swap jobs and hope you see them all in your lifetime? Read articles and hope you pick everything up correctly?
OP is giving a problem solving framework, it's not tied to specific solutions or tradeoffs. It's more of a method for thinking through a solution.
In more detail, the steps go like this.
Find as many solutions to a given problem as you can. Bad engineers run with the first solution that comes to mind, letting confirmation bias drive them.
Evaluate each solution for its costs and benefits. Imagine two steps in the future when the solution is implemented. What pains are there?
Search for creative new solutions that create win-win scenarios. That's riding the solution curve.
Given all viable scenarios, compare the costs and benefits against the quality measures for your specific context. Some projects value speed over precision. Some projects value performance over extensibility. Some solutions are easier to change later than others. This is choosing a specific point on the solution curve that best fits your context.
You can apply that method of problem solving to any problem, large or small. You don't need a ton of experience to practice it.
This comment made me realize what my mentor did years ago. He would have the entire team list all possible solutions to a problem, including the obviously bad ones, then have us whittle down the list based on pros and cons of each until we reached consensus on what to do. It was a teaching exercise and I didn’t realize it.
I’ve repeated it that exercise with junior engineers to great effect. Some catch on over time and start intuitively considering the trade offs of a few reasonable solutions to a problem; some don’t.
I never reflected on what he was doing there; thanks.
To add to that, some of the smartest "good" ideas come from "bad" ideas that people rejected out of hand, sometimes from unusual sources. It pays to do brainstorming thoroughly.
The best software engineers know when to solve a problem without using any code at all, like the classic "just do it manually" for a complicated task that is worth more than $x per task.
Definitely this. Nothing like learning from watching your solution hitting roadblocks (performance, extensibility etc) and reassessing your earlier approaches and again coming with a solution to address the newer challenges
You can also dig into the problems that existed when you got to the project, and try to work out how they came to be. Project forensics is a skill set unto itself. And listen to and help people on other projects at your company. You can see how their story arc goes and where it surprises you.
An intelligent man learns from his own mistakes. A wise man learns from the mistakes of others.
You can never truly be perfect, you can only approach it. Understanding your limitations is just as important as understanding your capabilities. Depth vs breadth is hard and you only have so much life to go so deep into so many domains.
- Identify and document possible solutions. Think hard about why each solution is good and each decision is bad.
- Within each solution, attempt to find common patterns in that problemspace (when working with Sharded databases, this approach brings these results, etc)
- Try to match your personal and organization requirements against the various patterns you find and the pros/cons you defined previously
During implementation
- Keep a running list of things that seem weird, or things that seem great, or things that turned out to be untrue
- Don't stop implementing, but for each thing you found that wasn't great in step 1, try to find a better way to approach that thing (while moving forward)
Post implementation
- Compile all the notes you made in pre/during into a manageable list of goods/bads.
- Use this to drive A.) iterations of your solution, and B.) future solutions
If you do this enough times, you'll have a pretty decent list of your experiences in a space, and you'll start to notice patterns as you try different things and become exposed to problems, their solutions and their tradeoffs. Eventually you won't even have to look at your previous notes very often, as you'll have built up a pretty decent amount of experience in various problemspaces such that you can predict what goes well/doesn't go well.
Also worth noting, this type of behavior doesn't really have to be applied to software, but will also gain you big points with future employers when interviewing. You'd be surprised to find how many candidates never stopped to reflect on the work they'd done, why it was sub-optimal, or how it could be corrected moving forward. Response like "We did it ____ way because that's how we always did it." Which, from a progress standpoint, is basically a none-answer.
Note: this is obviously just my opinion, and I'm no expert in "Becoming a Master of Stuff", but these are the methods that I have used and seen my peers use (in one form or another) over quite a long time. Sometimes not as obvious (doing these things in your head vs. writing them down), but the shape is always pretty similar. Ultimately I think being reflective is one of the best skills a person can possess (with respect to employment and I guess also relationships). Acting without thinking is reckless, I think.
I once was trying to design a validation framework. I thought it should work this way. The other lead thought it should work that way. So I proposed we write some sample code for these APIs and show it to the team. TDD with pseudocode before any of us knew what TDD was.
Only, I didn’t like having two to chose from. Something told me three would be better. So we paired, I wrote mine, then hers, then just made another one up on the spot. Once we agreed they were complete and implementable, I shopped them to the team.
About 2/3rds preferred the made up one. Including me. So that’s what I implemented.
In more detail, the steps go like this.
Find as many solutions to a given problem as you can. Bad engineers run with the first solution that comes to mind, letting confirmation bias drive them.
Evaluate each solution for its costs and benefits. Imagine two steps in the future when the solution is implemented. What pains are there?
Search for creative new solutions that create win-win scenarios. That's riding the solution curve.
Given all viable scenarios, compare the costs and benefits against the quality measures for your specific context. Some projects value speed over precision. Some projects value performance over extensibility. Some solutions are easier to change later than others. This is choosing a specific point on the solution curve that best fits your context.
You can apply that method of problem solving to any problem, large or small. You don't need a ton of experience to practice it.