Ask HN: How can I keep my Engineers properly fed with quality work?

[softprod]

Hi HN, I’ve been struggling with keeping my Software Engineering teams properly fed with good specs and enough work for them. I could use a little advice on what you all truly think is the right way to build software. This is an issue I’ve had for years and at multiple companies. I typically work in areas with high compliance burden like Healthcare or Fintech. If I have my product teams use lightweight tools like user stories and scrum/kanban, I get terrible products that haven’t been thought through. We miss subtleties, test scenarios, and getting my Engineers to fill in those details doesn’t seem to work. Maybe eventually you get a few Senior Engineers who learn to do it, but mostly they end up helping everyone else. If I go more heavyweight, and write large specs with use cases, state diagrams, sequence diagrams, gherkin BDD scenarios, we tend to get much better software but my org is really top heavy. I would end up needing more product people than developers, and training people to be able to do all that is really hard. So in the end, I personally always work 70hour weeks to fill in all the gaps from a product and engineering perspective, just to keep the process moving in the right direction. I recognize this is my failure in org design, and training, but what does good look like in your opinion? Thanks in advance.

[codingdave]

Personally, I run a Kanban board for product planning - the end of my board feeds into the beginning of the dev board, with the devs accepting the work onto their board so that if I missed anything or they need more info, it stays with me until it is complete enough that they can work it without questions. This speeds things up because I am not constantly just putting out fires due to poorly written cards.

I also find that product people tend to forget our purpose. PM 101: We say "what" needs done, Engineers say "how". So that is exactly what I write up. I don't diagram technical details, even though I also am an engineer and could do so... but I do get very tight on details of "what" should happen - "Click button a, do thing b, expect UI change c, etc." Cards typically outline a set of actions like that to comprise a feature, and I let the devs worry about whether it needs broken down into subtasks and shared amongst themselves or whether one dev will work it all. Again, my job is "what", theirs is "how".

Forgetting the goal of the product role often leads to product people who spend all their time in meetings micromanaging the dev process, instead of letting the engineers run free to do what they do. Which in turn leads to devs who complain about product people, and the whole team's morale tanks. If you can escape all that, the devs have autonomy to do their jobs and you have time to write good specs while keeping a lightweight process.

Edit: I should add that I don't set my specs in stone - devs are welcome to come to me and say, "Wouldn't it be better to do it this way?", and if their idea is better, I say yes.

[fumplethumb]

We’ve had some success with similar methods. In addition to the simplicity(it just “plugs into” the existing board), it’s nice to get metrics on the bottlenecks happening on the product side of things. If we can show that it historically takes 2 weeks to refine requirements, we can tighten up our estimates and shine a light on potential problems occurring there. If you only have metrics on the dev work, the more business-minded folks tend to focus improvements only on what they can measure.

I’ve also found that folding product workflows into the Kanban board gives the team more visibility and ownership over that end of the pipeline. We can observe how those items age before it ever gets to the dev team and swarm to fix things as a team earlier in the process.

[softprod]

Thanks for the detailed response. It makes sense that you have a kanban board specifically for product. So you feel like devs do a good job with the “how” once you have a good outline of set of actions?

[robofanatic]

IMO product people should not delve too much in “how” especially in large projects. Your role should be to communicate the requirements as clearly as possible to the engineering team and stay away from the “how” part as much as possible. I totally get the temptation of trying to influence the “how” part especially if your past life was in engineering. If engineers are not able to do the “how” then you need different engineers.

[codingdave]

Yep, I think the devs do a great job. If you can't have that confidence in your devs, as the sibling comment here says, "you need different engineers."

[satvikpendem]

Do you do all of the UX then (since you mentioned clicking on X and seeing Y output)? I've generally seen that the PM talks to the user and gathers the features that the user wants, the they hand it to the designer who thinks through the UX and creates the UI, that is then sent to the engineers to be implemented.

[codingdave]

We have a UX team - they have their own column on the planning board. They do work on features that require new patterns, but I write up cards directly if we already have patterns for the feature. No need to have UX burn effort on yet another field/form in the app just like the existing ones. So there is collaboration on who needs to do what for any given feature, and the board tracks it all.

[abyesilyurt]

How do you take care of timelines? A usual complaint I get from PMs is that, they are doing all the micromanaging because they need to provide some sort of dates to clients or upper management.

Do you maybe ask for an estimate on “what”-tickets?

[codingdave]

I don't take care of timelines. That isn't part of either "what" or "how". It is a whole different question of "when". I push back hard on even asking that question.

What I do instead is focus on priority. If the leadership thinks they need a date, I ask why. Often it ties to KPIs they need to hit, or promises they made up the chain, or sometimes even something real like: "We're out of funding in 4 months.". But once I know "why", I can tailor the "what" to focus tightly on that "why", and deliver. If they push for hard dates, then we're back into micromanagement to get the scope to hit the date, and we are wasting time again. So I don't measure effort... at all. I just let the team know what is most important, and we do the work.

The trick is to get leadership to trust that this works. Most of them are used to timelines, scrum cadences, 6 month roadmaps, etc. And if that works for them, great. But it isn't what I recommend for building an effective team who can deliver quickly while enjoying the work.

Edit: I always think of one more thing after I type up a comment... I can succeed in this scenario because I used to be an engineer. I know if something takes an hour vs. a day/week/month and can break things out accordingly. If a product person really does not have that sense, they'll need to ask about effort... but it can be at that high level of hour/week/day/month.

[aristofun]

> We miss subtleties, test scenarios,

This is something only 4x+ perfectionist types of engineers can do naturally.

They cost double the price tag, they need freedom and respect, you don’t attract them by leetcode interviews or arrogant managers but its the only way to get high quality product in reasonable time.

Otherwise you have no alternative but do it like everybody else — spend time and money on training, layers of management, elaborate system of acceptance tests, feedback loops etc.

Including training your users to live with your mediocre product and hope for the best.

[softprod]

I see. I’ve also seen there are some 4x+ Engineers who do this naturally but they are also really hard to find, along with being expensive as you’ve said. Which of these 2 paths do you think succesful tech companies rely on?

[dr_kiszonka]

I sound like the described engineer type (or so I was told), but without the 4x. I don't know if it helps at all, but I care deeply about my work and need 1) the environment to deliver results at least at the level of quality I expect from myself (in other words, don't make me produce crappy software), 2) technical challenges to help me grow, 3) meaningful projects.

[aristofun]

big companies don't have real choice, there are not that many 4x+ engineers on the market to cover all products and areas

one good example of a product type that can be (and is) made by a small team of high performers is Telegram app

I personally believe that inside a big company you still can and should have some top priority projects run by that kind of teams, if your management is smart and flexible enough

[dsattt]

There are plenty of 4x engineers available, you’re just bad at hiring.

[morbicer]

Similar experience as the sibling reply. I have plenty of colleagues that care about quality and details. Engineers are expected to fill the gaps on specs. Only few of them are 4x+ and we are all cheap because we live in central Europe.

The key is to have good company culture and pay above average (in local rates so 3x and more cheaper than USA). This give you great retention. People are loyal and aren't switching jobs every 2 years. Many of us are with the same company for 5+ years.

Which creates a sense of ownership. Give engineers freedom to make decisions [1], give them challenges. Not deadline challenges that lead to burnout. Foster collaboration and reward seniors teaching juniors. Maintain lean processes.

But truth to be told the dream starts to crumble. Our small to mid size company got acquired by a bigger one and the process bullshit and top down decisions start to creep up. I won't probably last long here but I am worried if I can find a company with a mindset we used to have.

[1] Perhaps controversial hot take: discard architects. Teams should own the technical design. If it's a huge project spanning multiple teams, host in-person designathon for a week. You should have staff engineers on board steering it, but involve seniors or even lower ranks. The people designing should be the same people who are going to implement it. Again, sense if ownership, lower communication overhead. If you are worried teams would stray in different ways, have guilds around core tech/architecture areas. Designs can be reviewed cross team. Let people join other teams for quarter or so to cross-pollinate ideas and best practices.

[dsattt]

They are not hard to find, you just reject them before you can see them.

[softprod]

I was a Software Developer for 10 years so I fully understand this industry can be toxic and sometimes exploits workers. I can't say that my recruitment practices are "great" but I do put a lot of effort into not rejecting people for dogmatic reasons.

[gardenhedge]

Do you pay above market rates? It sounds like you have a team of code monkeys (not meant to be offensive, but just a description) when you want senior software engineers and architects.

[softprod]

Most of my devs are in Europe and I pay slightly above market rates in those areas. Perhaps not enough though.

[dsattt]

You hired code monkeys.

[softprod]

Some of them yes possibly. I can control some of the budget but not all of it. I agree that the ideal would be to pay significantly above market and buy my way out of these problems.

[enormousness]

We use architects for what you've described. It's a distinct technical track from developers and from team leaders. They're usually senior developers with outstanding experience in multiple technologies. They deal with technical design, specification and documentation. They sit in-between the product owner and the team in the story flow. The PO feeds the team with user stories and priorities, the architect fills in the technical spec, developers choose the implementation and provide the time estimates, QA uses the spec to test.

[gardenhedge]

Could you provide some more detail on what the architect does versus the developer? Is my example below along the right track?

e.g.

* PO says they want a way for customers to book an appointment on X product

* architect identifies where that change should be made, how it integrates with the rest of the system, identifies any non-functional requirements that need to be addressed, identifies any third party dependencies, what docs should be updated,

* dev writes and tests the code, supports through the testing, release and post-release phases

[JonChesterfield]

I'm confused by the problem of finding enough work to keep engineers busy.

For one thing there is always more work. For another engineers should be totally capable of finding it for themselves.

Some industries really like specifications. Others don't bother with them.

[thebruce87m]

I interpret it as ”useful work”, progressing toward a successful product, as opposed to rearranging the deck chairs on the Titanic.

[softprod]

Right, that’s how I meant it.

[austin-cheney]

Two things: rewards and separation of business concerns.

First, you need to properly motivate your people to take maximum initiative to build things. If there is time to spare there is time to improve internal automation and refactor existing internal software processes. Do it, because that churn is normally hard to financially justify but it’s how you build superior software.

Also, use that time to make massive advancements in documentation. The documentation should be human readable and not specs. Specs are technical documents created for either planning purposes or conformity in the wild.

You need to reward your people for small successes that increase automation, reduce tech debt, and so on. Rewards can be things like recognition programs, points for periodic evaluations, lunch coupons, PTO, awards, and so forth.

Separate out business concerns so that the developers are not churning on product decisions unless the developers are creating new original software tools. When managing open source projects there are many product decisions to be made but most of the energy is actually documentation and maintenance. If necessary bring a product owner into the mix to advise your developers on these internal initiatives.

[simne]

First, you need to understand and accept fact, this is not engineering question, but economy.

From this economy point of view, you could start your looong journey to project management.

Or you could hire some person, who will done this work for you, but for quality, you must obey what he say.

I will tell you some words to google, but without deep understanding of economy this will be mostly waste of time.

But to show I'm serious, short list (not complete, but most important things):

1. Goldratt Theory of constraints.

2. PERT (project management), and critical path PM.

3. Business models.

Good luck. And sure, you could contact me if interested in more details, or who knows may you decide to hire me.

[mattmanser]

If you're constantly having this problem, it sounds like you're the problem.

You're gate keeping the end-users so the engineers can't become domain experts, albeit maybe inadvertently.

Good engineers will become domain experts naturally from talking to clients, usually experts far beyond the actual business users. That is, unless someone's interposing themselves between the engineers and the users.

It sounds like you are the person interposing, and you are the reason the engineers are not becoming domain experts.

That or you've always worked with bad engineers.

[softprod]

I haven't really thought of that as a possibility and I'll accept it may be true. I definitely wouldn't do that on purpose so I will have to take a closer look at how I treat situations like this.

[extasia]

Not a senior engineer, but my instinct is that you need to train people s.t you are confident delegating at least parts of your current role to them.

What's stopping you from handpicking a (willing) senior and having them shadow you / learn to do parts of the planning?

[softprod]

Good point. I historically have been weak on training, probably because it feels like so much work. I should look into investing more time and money into it. Sounds like maybe lack of training is a primary issue.

[bomewish]

You should be able to tell from the calibre of your people if they can be “trained” to actually do what you’re asking right? It sounds like a combination of experience, intelligence, and conscientiousness. The first can be changed but the latter two hard to do much about. That may not be resolvable as a “more training” issue.

[Racing0461]

> If I go more heavyweight, and write large specs with use cases, state diagrams, sequence diagrams, gherkin BDD scenarios, we tend to get much better software.

This is how it should work. I don't see how having a few PMs do this is top down heavy?