[dekhn]

One thing I've learned is that even when folks think their problem can only be solved in a particular way (fast interconnect to implement the underlying physics) there is almost always another way, that is cheaper and solves the problem, mainly by applying cleverer ideas.

I'll give (yet another) AMBER example. At some point in the past AMBER really only scaled on fast interconnects. But then somebody realized the data being passed around could be compressed before transmit and then decompressed on the other end- all faster than it could be sent over the wire. Once the code was rewritten, the resulting engine scaled better- on all platforms, including ones that had wimply (switched gigabit) interconnect. It reduced the cost of doing the same experiments significantly, by making it possible to run identical problems on less/cheaper hardware.

Second- I really do know a fair amount in this field, having worked on both AMBER on supercomputers (with strong scaling) and Folding@Home (which explicitly demonstrated that many protein folding problems never needed a "supercomputer").

[saltcured]

I do not know much about your field of molecular dynamics. But, it is my lay understanding that it tends to have aspects of sparse models in space, almost like a finite-element model in civil engineering. Upon this, you have higher level equations and geometry to model forces or energy transfer between atoms. It may involve quadratic search for pairwise interactions and possibly spatial search trees like kdtrees to find nearby objects. Is that about right? And protein folding is, as I understand it, high throughput because it is a vast search or optimization problem on very small models.

Compared with fluid dynamics, I think your problem domain has much higher algorithmic complexity per stored byte of model data. Rather than representing a set of atoms or other particles, typical fluid simulations represent regions of space with a fixed set of per-location scalar or vector measurements. A region is updated based on a function that always views the same set of neighbor regions. Storage and compute size scales with the spatial volume and resolution, not with the amount of matter being simulated. These other problems are closer in spirit to convolution over a dense matrix, which often has so few compute cycles per byte that it is just bandwidth-limited in ripping through the matrix and updating values. But, due to the multiple dimensions, it is also ugly traversals rather than a simple linear streaming problem.