[tgflynn]

If it takes as much effort to write the unit tests as it does to implement the solution then the problem was trivial to begin with, but that's probably true of 90% of all programming work.

It might be more useful for harder problems but there are already a number of sites addressing those : TopCoder, kaggle, Innocentive, etc.

[milohoffman]

One thing on my "to-do" list is adding a test suite builder, that allows you to specify inputs and outputs and then generates the test code for you. That would allow people who can't code or just aren't familiar with the syntax of a particular language/framework to still post contracts.

[thebiglebrewski]

That's an amazing idea.

[lifeisstillgood]

Cucumber and related "natural language tests" were supposed to do this - allow non-programmers to program tests.

Turns out everyone who writes a cucumber test is a programmer writing their own unit tests or a whole batch for everyone else.

I do think there is mileage in wufoo / VWO style webstie testing - record a series of macros and replay them each day against the site - let people know when something changes / does not work. ie a regular scheduled sign up with specific credit card then a cancellation - test that the whole sign up / email thing is still working.

[ericclemmons]

I started working on exactly your last suggestion. We have employees that were doing this, I said it was a waste of time, they said to have some of our devs in India do it.

Instead, I created "Amir" which is effectively a Selenium/CasperJS runner performing pre-recorded tasks each day and checking for outliers. Once it fails, emails go out.