[chubot]

Hm so when I am writing web servers, I typically have a bash script that simply does curl on a bunch of URLs. I just run it by hand and make sure it stops on the last failure with 'set -o errexit'. I divide it up into bash functions so I can run the set of tests I want, and then string them all together with a top level function.

It's ultra-minimal, but that is the point. My job is to deliver working software (relatively quickly), not necessarily beautiful tests.

And I actually should say I do "system tests" in bash, not necessarily unit tests. But I START with system tests. My workflow is now like this:

    - write SYSTEM tests in bash first.  This helps me plan what I want to get working.
    - write the code (It's like TDD but with system tests)
    - sometimes there is some complicated logic I need to test.  Then I switch to unit tests in Python, JS, R, C++, etc.
    - when I fix bugs later, I tend to reproduce it with a system test.  This makes bug fixing easy.  Then I write a unit test to zero in on the broken part, and sure it never comes back.

I probably save 70% lines of test code, while delivering the same level of quality. A system test can be fast and can substitute for a lot of unit tests, and it helps you not ossify your code structure too early.

I don't have an example of that online, but you can see 880 lines of shell in the root dir here: https://github.com/google/rappor

Granted, this is more like scientific / batch code than web / backend stuff, but I assure you I follow the same strategy in the latter case. It's basically using Unix as leverage.

One thing I would notice about the scripts is how the last line is "$@". That runs the function $0 with the args $1 ... $9. This is a very handy trick that helps keep everything modular and independently executable.

I use shell scripts for all these things:

    - building code (in C, JS, Python, or any language)
    - testing code (running unit tests in multiple languages, and also system tests, constructing test data)      
    - deploying code
    - sys admin; setting up servers (e.g. setup.sh in the repo), backup, managing log files
    - building docs; making static web sites   
    - building config files and data files (e.g. lists of URLs)

My assertion is that, combined, these things take up MORE time than actually writing code! If you can speed them up, you will be more productive at "coding" overall.

In addition, using the same tool (bash) for all of them makes you more productive (once you have climbed the admittedly substantial learning curve). I'm not a fan of all these language-specific build tools now. I don't even use make that much; my first choice is bash!

Here is one concrete example of shell minimalism, which helps you get things done faster. Think about the common situation of having a {star}_test.py files in many directories, and you want to run them all together as a test suite.

I am embarrassed to say that at one point I spent 2 or 3 days researching and implementing some kind of Python test runner (this was over 10 years ago). I think people do it now with "nose" or something.

Here is my preferred solution:

find . -name {star}_test.py | sh -x -e

Or if you need parallelism, something like:

find . -name {star}_test.py | xargs --verbose -n 1 -P 10 -- sh -c

({star} should be asterisk; HN is not letting me escape those)

That's it. It doesn't necessarily give you pretty output, but it's what you need to deliver working software. Add one line to a bash script and be done with it, rather than getting distracted by yak shaving.

The shell scripts may not look that impressive until you think about what they actually save -- what common alternatives are. One line CAN actually save you 2 or 3 days of time; this is not the only time I've experienced this.

The overall point is that I used to spend a lot of time on tooling and automation (yak shaving), because I wanted a certain level of code quality and iteration speed. But now I do most of those things with a small, sharp tool -- bash -- and I find it saves a lot of time. I get in the zone and don't get distracted from the main task.

[notduncansmith]

So it looks to me like the main case you use bash scripts for is integration tests, and you simply check for whether there is an error (as opposed to checking the output). Do you run any sort of assertions on the resulting data (for instance, POST {x} followed by GET {x} should return {x})?

As much as I appreciate minimalism (and the idea of tests in Bash do seem alluring), for anything involving large JSON structures or wanting to know what specifically about a particular test run went wrong, it seems like Bash would fall short. I'd love for you tell me I'm wrong though.

[chubot]

I do smoke tests in bash for sure, but writing exhaustive test cases may be better done with unit tests in the language.

You can do something like:

  curl http://localhost/ > actual.json
  diff -u actual.json <<EOF
  {"expected": "json"}
  EOF
  check $? -eq 0

Where check() is a simple wrapper around "test". Or you could write a 2 line "check-eq" function that does the diff too, and that would get you pretty far.

I guess the meta-point is that bash is "just enough". My goal is to deliver working software, and a smoke test written in 10 lines of bash can help tremendously (i.e. reduce trivial errors that cause havoc with a deployment.)

I actually started a bash test framework, probably 2-3 years ago. But honestly, I haven't needed it as much as I thought. Sometimes I copy and paste some functions like "check" into new projects. It ends up being different for each project, so it's perhaps not worth generalizing.

The point is to be lazy, and put off all non-essential tasks, and writing a general test framework for bash proved to be something non-essential. There are existing bash test frameworks out there for sure, but I haven't really found the need to check them out.

Hope that helps.

[notduncansmith]

It does actually, thank you very much for taking the time to reply in such detail. Bash testing (or rather, zsh in my case) is definitely something I'll have to consider moving forward.