[pmwhite]

The second. In the given example, if skilled programmers cannot find the bug in a couple dozen lines of code within an hour, it is not an Agile vs Better Agile vs non-Agile question. It is a question of whether there is a member on your team who has the "toolkit" to test concurrency scenarios sufficiently well to find this kind of bug.

Of course, the hard part is recognizing when to use formal methods and when not to. I do wonder if someone who had used TLA+ (frex) in the past could look at this code and know that now is the time. Or not.

[gbacon]

At the bottom of JavaUnitTestChallengeSolved[0], William Underwood posted the test below and claims that it always catches the `notify()` problem. He did not mention using TLA+.

    public static void testConcurrency() throws InterruptedException {
      final int[] threadsFinished = new int[]{0};
      final BoundedBuffer buffer = new BoundedBuffer();

      Thread thread1 = new Thread() {
        public void run() {
          synchronized (buffer) {
            synchronized (this) {
              this.notifyAll();
            }
            try {
              buffer.wait();
            } catch (InterruptedException e) {}
            threadsFinished[0]++;
          }
        }
      };
      Thread thread2 = new Thread() {
        public void run() {
          synchronized (buffer) {
            synchronized (this) {
              this.notifyAll();
            }
            try {
              buffer.wait();
            } catch (InterruptedException e) {}
            threadsFinished[0]++;
          }
        }
      };

      synchronized (thread1) {
        thread1.start();
        thread1.wait();
      }
      synchronized (thread2) {
        thread2.start();
        thread2.wait();
      }
      synchronized (buffer) {
        buffer.put("");
        thread1.join(1000);
        thread2.join(1000);
      }
      assertEquals("Not all test threads were awoken!", 2, threadsFinished[0]);
    }

[0]: http://wiki.c2.com/?JavaUnitTestChallengeSolved

[hwayne]

That's just testing that all of the test threads are awake after they run once. It only does one buffer operation (the `put` in `synchronized buffer`), and we trivially know it's "broken" if there are only puts and no gets. It's also using two threads for a buffer length of 4. This isn't actually surfacing the bug, just something that looks somewhat like the bug if you already know what the bug is.

[EdSharkey]

I like your "toolkit" metaphor for test-ability in any case.

If you had asked me a year ago if you could Test-Drive UX development, I would have said no, UX tests are too brittle, and that one should just visually inspect them and focus on testing the data flowing to the view and commands flowing from the view.

But then I got some exposure to the Jest framework for testing and was blown away in that they've automated the acceptance testing and automated accepting changes to expected outputs in order to mitigate the brittleness problem.

And before that I learned about the Quixote framework for CSS testing and learned I could TDD my CSS code (to some extent), making that development more deterministic.

These tools completely transformed my beliefs about what the UX development process could be.