[rs23296008n1]

Many moons ago when Beowulf clusters were still new, I remember a project where I was given a months worth of then new cluster time to spend. Due to a delay we had to wait a few months before we could exercise it. One weekend I was playing around with some junk computers I'd assembled for a LAN party. Long story short, I tried out a portion of the project on that LAN, got a useful result then we ran the rest of the project on that overall system and joined more machines. We completed the project using no special machines, often just idle machines added and removed upon their availability. Along the way we rewrote the entire project code as a set of modules then used python to orchestrate them.

None of this is extraordinary now but the result was we reduced multiple times the budget requirements and processing times just using code improvements. A lot of times, the head-on solution just needed optimisation. Sideways improvements also helped such as small optimisations also helped. The more exotic equipment is still useful but its an accelerator.

One last thing: we received a LOT of grief and criticism for our approach. There was peer pressure to use particular solution types even though they were wildly inappropriate. We had funding pulled or threatened to be pulled by some and other backers. One lessons we learnt includes: Don't underestimate how vested certain interests are for the use of various toolkits. "Use this or else!" is the not-so-subtle threat.

I'm so glad to not rely on only academic work now.