[ottoflux]

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.

[jeremyjh]

Agreed. QA specialists are there to think about what the engineer didn't think about. Unless the engineer is incompetent or the organization is broken, the engineer has already written tests for everything they could think of, but they can't think of everything.

More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time.

Like everything else though, its contextual. Complexity of domain, surface area and age of product, depth of experience on team and consequences of failure are all so variable that there cannot be only one answer.

I have done it both ways for many years. I have worked on teams where QA is a frustrating nuisance, and teams where they were critical to success. I have worked on teams that did pretty good without them, and probably those were the highest throughput, most productive teams because the engineers were forced to own all the consequences - every bug they shipped was a production issue they were immediately forced to track down and resolve.

But those were very small teams, and eventually I was the only founding engineer left on the team and far too many mistakes by other people made it to my desk because I was the only person who could find them in review or track them down quickly in production. That was when I started hiring QA people.

[MoreQARespect]

Ive almost never worked on a project where there was the right number of QAs who were doing the right thing.

Usually there either arent any in which case bugs get missed or there are 5 very cheap ones running mindless scripts who are standing in for the devs' inability or unwillingness to write decent automated tests but dont catch the really deep level thorny stuff.

[ghywertelling]

QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it. The average people who do don't do a good job.

[locknitpicker]

> QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it.

Hard disagree. QAs' main tasks are acceptance testing and V&V. This means their work is to act as independent parties whose role is to verify that the results are indeed being delivered. Their importance is up there next to Product Owners.

The problem is that some QA roles are relegated to running the same old test scripts and do some bullshit manual exploratory tests that catch and assert nothing. It's the ultimate bullshit job as it's performative. This is clear in the way that LLM agents are seen as threatening the very existence of the role.

[ricardobayes]

Interesting username

[anothereng]

i know right hahah

[9wzYQbTYsAIc]

> More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time.

Personal liability and professional insurance works for all the actual “professions” in the US, to some extent, right?

It might be time to start the considerations for professional licensing for platform scale or commercially published software.

[rfrey]

Real engineering disciplines also have dedicated QA and test engineers.

[ivan_gammel]

More like certified products. New ISO standard may require professional liability for software products, which will be adopted as requirement by big consumers and will pull the industry into certification loop, because insurers will ask for it. This will obviously put a high entry barrier to many product categories, slowing down innovation.

[9wzYQbTYsAIc]

Yes, but slowing down to avoid hazards is sometimes important.

Medical devices and such are the only places I’d expect to see the need for certified products. By extension, in the new era, we really ought only expect certified software where we expect a duty to care from the software system (or any other assigned duty)

[ivan_gammel]

In development of medical devices existing quality controls are already working well, right?

[9wzYQbTYsAIc]

My point exactly, embedded devices are the closest software gets to actually being built by licensed engineers. The expectation can often be that you are an electrical engineer by training, where licensure is a viable path, unlike in software engineering.

[_doctor_love]

Couldn't agree more strongly. The best QA people have the old-school “test to break” mentality. They do weird things, like pulling the network cable out of their machine mid-transaction, or kicking off a series of performance scripts and then powering off servers in a distributed system just to see what happens.

When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at Apple told me that being a QA person (in the industry) was once seen as a high-trust position. He was sad at what it became.

[ffsm8]

When I got into software that employer was pretty small (50 people overall I think).

Their approach to QA was to have this be a optional thing the service people could do.

It worked surprisingly well, with the caveat that they never created regression tests.

The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team

[canpan]

Yes, QA is important. My code will always "work" in that everything I tested is bug free. But having someone other test, especially someone who knows the service is gold.

But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet.

My most impressive QA experience where when I helped out a famous Japanese gaming company. They tested things like press multiple buttons in the same frame and see my code crash.

[darth_aardvark]

> But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet.

This was my sole experience at the one place I worked with an internal QA team. They absolutely could never find bugs that devs missed, often mis-marked ones that didn't exist, and failed to find obvious edge cases that did exist.

Multiple devs fired because the CEO believed the QA over the engineering team; if they marked a bug as present, it was the engineer's fault for writing it. If they didn't catch a bug that made it to prod, it was the engineer's fault for not including it in the test plan. They represented nothing but red tape and provided no value.

Good QA sounds great! I'd love to know what that's like someday! It'd be great to have someone testing my code and finding breakages I missed! I'm only slightly (incredibly) bitter about my bad experience with its implementation.

[Ekaros]

I do think the type of testing where QA just follows pre-generated script has place. But it is about long term regression. The first round absolutely should not find anything. But with complex system it also should find nothing in a year or three or five years... Offloading this to dedicate resource could be useful in certain industries.

[canpan]

I did not think of that. Maybe for some industries, it might make sense. But if I want a regression test, I would probably set it up as automated test. In the case I mentioned above it was the only test beside my own for a new service.

[philk10]

Not really that impressive, that's Testing Quick Attacks 101

[jghn]

The tension is that QA is important. But most QA practitioners are not good. The world is filled with QA people who couldn't make it as an SWE and now are button pushers. But high quality QA people are amazing. These are the ones who understand how to break apart a system, push it to its limits, and engineer the quality plan.

This is an area where I expect AI to create a bimodal future. The smaller group of high quality QA people will now be able to offload the activity to agents instead of the QA drones. They'll still be worth their weight in gold, whereas the drones will be redundant.

[malfist]

I've worked with a lot of QA folks that just repackage up the unit tests the dev already writes. And I've met a few that strikes out on their own and comes up with real tests.

The latter is much more high touch, but they're often worth their weight in gold. The former is kinda pointless.

[jghn]

Exactly. I think AI tooling will make that good group even more effective. And it will make the bad group even more pointless.

[LatencyKills]

Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)

[MoreQARespect]

Really? The best people I worked with were never QA.

Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.

I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people.

I think it's overdue an overhaul much like the sysadmin->devops transition.

[LatencyKills]

We have differing experiences, which shouldn't be surprising. My example explicitly referred to someone who was a good engineer who enjoyed the QA role.

This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.

[stiglitz]

In the Windows team (back before the test org was decimated) I saw the described "class divide". Anybody who was good enough would switch from SDET to SDE [disclaimer: obviously there were some isolated exceptions]. The test team produced reams of crappy test frameworks, each of which seemed like a "proving project" for its creators to show they could be competent SDEs. After the Great Decimation my dev team took ownership of many such frameworks and it was a total boondoggle; we wasted years trying (and mostly failing) to sort through the crappy test code.

This was all unfortunate, and I agree in principle with having a separate test org, but in Windows the culture unfortunately seemed to be built around testers as second-class software developers.

[LatencyKills]

I spent most of my time working on Visual Studio (in the Boston time frame) so we got to interact with pretty much every team. I absolutely hated interacting with the Windows team. Everything was a fight for no reason.

As I said above, everyone has their own experiences but the QA folks I worked with at MS were fantastic.

Not sure if you're aware but Dave Plumber now has a really good YT channel [0] where he talks about MS back in those days. It's a fun walk down memory lane.

[0]: https://www.youtube.com/@DavesGarage

[saghm]

> Really? The best people I worked with were never QA.

> Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.

That sort of seems circular. If they're not respected or paid well, of course most of the talented people would not want to remain in QA, and eventually you'd just have mediocre QA. That doesn't really give you any insight into whether high quality QA would be useful though.

(edit: I see now that's basically the point you're trying to make, so I guess we're in agreement)

[pixl97]

I mean the people that come up thru QA may be the best while getting enough time in the company to go to a position that pays.

But yea, so many companies cheap their QA and then wonders why their QA sucks.

[z500]

We don't have dedicated QA at my job, but personally I'm thrilled when a coworker finds a bug in my code. It's much more preferable to the customer finding it

[D-Coder]

Yes! I had a QA person apologize once for finding a bug in my code. I thanked him for not letting it make me look like an idiot.

[dotancohen]

  > They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.

Did you try with 2 QA people? 65536 QA people? 0 QA people? NULL QA people?

[locknitpicker]

> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist.

I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?

[efsavage]

Great QA people are rarer than great developers, and potentially even more valuable.

[pphysch]

Yes, QA and test engineers tend to have a better ability to specify the correct behavior of a system than anyone else. It's literally their job.

This is a big asset in the current paradigm where a LLM agent can effectively implement behavior only when given a robust test suite to iterate against.

[stingraycharles]

> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering”

I don’t understand the reasoning here why QA shouldn’t be engineering.

[9wzYQbTYsAIc]

> I don’t understand the reasoning here why QA shouldn’t be engineering.

Who watches the watcher, right?

That aside, the core idea is the same as the principles of independent audit, peer review, or even simply just specialization.

Red team / Blue team?

[stingraycharles]

Yes but both the red team and blue team would still be engineering.

[IAmBroom]

The red team and blue team should not share supervisors.

Nor, in the case of QA, should the audit team be engineers trained to act and think like the ones who wrote the software. A fresh perspective is useful.

But in the long run, supervisory independence is the real deal. I know of a QA manager who shut down an entire factory's output until a major safety issue (that had been kicked down the road several times) was addressed. It took chutzpah, and serious power, to do that. The Dir. of Engrg. would NEVER have allowed it.

[9wzYQbTYsAIc]

Yes, but police and military are both law enforcement, on one level, but each are very different from the other.

Even the military have police, right?

edit: ultimately, it comes down to the importance of independent audit, the builders and the breaker/fixers are very different groups in engineering.

[liampulles]

Frankly, calling software development engineering is quite debatable. We should be calling less things engineering that aren't actually engineering qualifications.

[bigfatkitten]

Being a branch of engineering implies a certain level of professionalism and accountability that the software development community actively resists.

[rfrey]

Engineering like the guy in the booth at a show is a sound engineer. Talented: check; challenging work: check; valuable: check; creative: check. "Engineering" like designing a building, bridge, or power line? Nope.

It's not a protected term in the US so it's jarring to those of us living where it is.

[Perepiska]

> with their bugs

This is funny.

[mememememememo]

It should be under engineering but as a seperate skill/role. To ship fast you want QA aligned with milestones, sprints etc. You want QA to feel the deadlines, and then like an Eng push up if things aren't realistic.

What I have seen before with QA is a queue like system. Hand to QA. get back in 3 days. Not the QA persons fault. It is the setup if the org that is wrong.

Imagine if you had a code review role and code review team and you had to wait to get your code OKed. We wouldn't put up with it.

QA teams seperate from Eng worked when the software was burned to CD rather than uploaded to s3.