[nickparker]

This paper[1] also currently on the front page is a great intro to MSP if, like me, you aren't familiar with the paradigm.

I'd be interested in hearing some exameple applications of this, if anyone here uses MSP on streams in serious applications. Killing runtime overhead like this seems very powerful intuitively, but use cases where it would have a significant impact don't immediately jump to mind for me.

[1]: http://www.cs.rice.edu/~taha/publications/journal/dspg04a.pd...

[noelwelsh]

I'm currently working on a new streaming system for a big data real-time-ish application (which is why we can't use existing libraries). We've come up with a design that is very similar to that described in the paper, as I understand it on a quick skim, but we don't use MSP and hence can't perform deforestation. We did actually consider using LMS but decided we weren't justified in doing so without getting some benchmarks first. I think we'll hit our targets without going down the staging route, but it's good to have prior work to build on if we need to!

Linear algebra is the other obvious-to-me application of this. Linear algebra libraries typically either create a lot of unnecessary intermediate arrays or provide ugly imperative APIs (hi BLAS!) and force that busy work onto the programmer. Deforestation is an excellent way to regain performance while still maintaining ease of use. Applications include deep learning, and this is broadly the approach taken by TensorFlow and other libraries.

[nickparker]

It's interesting that you bring up Lin Alg... I do a lot of computational geometry work and I've always rolled my own supporting libraries from essentially nothing because I (likely naively) prefer that level of control vs learning some mega-framework's abstractions of choice.

It sounds like I need to read more about MSP and maybe start using it.