Ask HN: Would you join a startup that doesn't test? Are tests impediments?

[dootdoot]

Met this great team at a startup running a (quite successful) webapp on a cloud hosting provider. They don't test their code at all (and instead run through tests manually before they push to production).

They say it's because they wanted to put something out there rather than focus on testing. Is that legit?

I like them a lot, but it makes me nervous that they don't have any test framework in place. I think they're open to testing now they are more stable. Thoughts?

[kennethtilton]

TDD is overrated and overused, with no regard for the cost of developing/maintaining the test suite. You should join them -- you will probably learn something about building correct software, such as "automated regression testing is not the only way". Ironically, tonight I have fired up the RT I use for one subsystem because it needs work. ie, RT has its place, but the TDD crowd thinks every place is that place. btw, the idea that you would not join a group you like because they do not use TDD confirms what I have come to suspect from interviewing more than a few Rails folks: TDD is a bit of a cult, a tail now wagging the dog of how its adherents think about programming. TDD can be great, but it has its place, limits, and cost.

[humbledrone]

You rail about TDD, but the OP is talking about tests in general, quoting, "They don't test their code at all." I personally thing that TDD is totally backwards and braindead, but unit tests... you just have to have them. The only acceptable time to eschew basic unit tests is if you plan to throw the code away SOON (and have a blood pact about killing it for sure).

[kennethtilton]

Ha-ha, quote further! "They don't test their code at all (and instead run through tests manually..". Actually, I meant to mention that as another sign of cultdom: the verb "test" now means "automated test". Manually testing does not count, nor do any of the other ways we can get to correctnes without TDD.

[jdietrich]

Your conflation of TDD with automated testing is not helpful.

As software developers, the essence of our profession is to automate repetitive tasks. If you have a rigorous manual testing methodology for your software, it should be a trivial task to turn that test specification into automated tests. The absence of a suite of automated tests is a major red flag in any non-trivial software project - it strongly suggests that either the developers do not care about quality, that they do not understand their work well enough to manage quality in a systematic way, or that they are too haphazard and disorganised to manage their project appropriately.

[kennethtilton]

I am just waiting for someone to ask, "How else can one achieve correctness?" :)

But cults do not do that, they just accuse non-believers, in this case, that they do not care about correctness.

That's like saying Messner and Habeler did not care about safety because they climbed extreme pitches unroped where other pairs roped up and moved one at a time. But M&H were very good climbers and got up and down twice as fast as the others, who doubled their risk of the weather turning on them by being "safe". If you know your mountaineering, the latter is usually how folks die on mountains.

[zura]

Automated testing is not necessary TDD.

[kennethtilton]

OK, I'll stop saying TDD. :)

[dootdoot]

Yeah, it's not about TDD, it's about having your testing be a non-automated checklist (without unit tests where useful)

[kennethtilton]

Granted I am reading between the lines, but I think the OP was indeed concerned that this great new team he found was not a member of the TDD cult. As i just threaded separately, they equated testing with automated testing, and that set of my cult alarm.

[dootdoot]

as the actual OP, I can definitely say I wasn't talking about TDD. I mean integration tests like "Hey, we aren't double charging this person" or tests to make sure that bugs stay fixed. It doesn't matter to me if you write tests before or after the fact, you just should be testing so you can have some sort of sense of correctness.

[kennethtilton]

That's fine. My point was that for you automated testing is a litmus test that turned you against a successful group of great guys. What I do in those situations is check my presuppositions. Maybe I can learn something.

Did you ask them about their practice? Did you ask how often they had to roll back deployments, or at least take a hit on something automated testing would have caught?

[dootdoot]

That makes sense. I've found automated testing to be incredibly useful and just necessary to be able to really rework code well. Not using automated testing and having a really fast release cycle seems like a recipe for a lot of pain (and they said that testing was taking them forever).

I asked them about their testing - turns out they were looking to have someone help them develop a way to test their software (so at least they want to do it).

I wish I'd asked them the two questions you listed at the end, they are much better questions than a bland "do you test?". They did break the build some times.

[acesubido]

If they're only starting out, it would be okay if there would be no testing at first, but as the months roll by, they'll need testing sooner than they think.

If they've been working at it for a while, joining them is going to be a lot hard work. For one, it can be a warning sign, in the first place, their code will be hard to test. Dependencies will be out of this world, who knows where they put their business logic, random classes doing random things, random database session handling, god objects, and other elaborate hard-to-maintain hack stacks.

If you still want to join them, introduce testing to them by creating tests for one small part of their stack, it will easily show off the merits of testing:

- It's not just about putting something out there faster, it's also about failing business assumptions and the capability for your code to iterate/pivot quickly. Testing helps you in that aspect. Spend 10 minutes running the entire stack and tracing where a bug occurred, or just run your tests and find out what went wrong.

- The great thing about automated tests is that whenever I make huge changes in my code and the tests still run it means I can confidently say I didn't break anything at all. If I did, then I can easily know where and how, and I don't have to run through the entire app. I could easily isolate a single piece of my stack and test directly.

- Testing forces the team to write maintainable software.

- Another amazing meta-feature about tests is that, whenever you're debugging, it takes a huge chunk of business logic out from your cognition. You don't need to carry the extra cognitive load that 'x' part of your stack be able to 'y'.

Anyway, join them not for the sake of their code, join them if you just love what they're doing. You'll work through it.

[stevoo]

Another important argument for tests is that as people like your self join the company they will have 10x a harder time doing any change to the source as they will not now how this might impact the whole project. It might break somewhere that you do not know it will. (first level experience on this). A good test base, will make your live a lot easier as you go along.

[humbledrone]

Hell no. Hacking together a crappy throwaway minimum viable product with no tests is one thing, but if this company is "quite successful" (as you say) it is probably past the time where they should have started writing tests.

In the even slightly long term, tests don't slow things down, they keep things fast. It is MUCH easier to change and maintain code with good tests. Think about it this way: when you write code, you damn well better be testing that it works as you write it. Why not test it with other code, and save that so that other people don't have to do it over again? Manual testing is slow, error prone, and very expensive. Better use it judiciously instead of making people track down stupid programming errors that could have been caught in 7 seconds with a decent unit test suite.

[kennethtilton]

Get over it. They are successful without TDD. It is extremely possible, and then one does not have to slave over a hundred lines of tests when one decides to write ten lines of code. TDD has its place, but it is a relatively small place (and then, yes, it is worth its weight in gold -- I do it often).

[palidanx]

When starting an mvp, you are bleeding (usually) bootstrapped cash. In that scenario the priority is really to get the product out the door, and some stable revenue stream. Testing is often put the wayside just to survive.

In an ideal software development scenario where you have a budget, then yes, testing should be there in the beginning.

But if it is a start-up, I'm tempted to give slack for the lack of a framework. And it seems like if you come in, you might be the person to lead the testing framework which would be a good thing.

[argonaut]

Facebook didn't have tests for quite a while.

I talked to a very hot startup (100M+ valuation, very solid business model) that everyone here has heard of, and they had no tests at all.

Don't discount a company because of their testing situation. Heck, maybe you'll be the first engineer to start writing tests!

[Wezc]

With my friends (we are french) we always say: "Tester c'est douter" meaning something like: "If you test it's because you are not sure, you doubt" Or "Si ca compile, ca marche !" = "If this code compile, it works!"

[dootdoot]

unless you use a dynamic language :P

[olegh123]

no. Huge red flag.