Hacker News new | ask | show | jobs
by absconditus 5540 days ago
We get it. The title is wrong. I am rather disappointed by the low quality of the discussion here about a rather interesting article.
1 comments

Fair enough. I'm reluctant to say too much since I actually work on Streams (hence my authoritarian corrections), but I'm a research staff member, not a marketing person.

I suppose I do feel comfortable giving my personal take on it, and why I find working on Streams exciting: we're designing a language and runtime system for a new programming model. Not just a new language (which we do have), but a new programming model. The way you write programs changes when you have essentially infinite streams of data coming through your system - you have to think in terms of operators that transform or filter individual pieces, and you have to keep in mind its inherent distributed nature. That is, you may write a chain of operators to process your data, but each operator can run in parallel with the others, even though you will probably design your application thinking of one particular piece of data marching through the system. The runtime system itself is, of course, distributed and fast.

To get an overview of this programming model, take a look at this paper: "SPADE: The System S Declarative Stream Processing Engine": http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.160...

Just keep in mind two things: the newest iteration of the language is called Streams Processing Language (SPL), and that it is a complete rewrite of Spade.

I suppose I should put in the disclaimer that these are my views, and I do not represent IBM.

To be a bit blunt, that article with a heavily misleading title totally disappointed me. I was expecting to glean some insights there on how they applied certain kind of technologies into a vertical market -- nothing but a few marketing pitch.

Your post got me even more confused: what exactly new programming models? I understand there may be an influx of sensor data coming in and your apps have to be able to cope with them --- do specific data analysis, maybe some kinds of expert systems, and generate possible suggestions. But how is that different from existing systems, say, nuclear station operation system, high-frequency trading systems, to name a few.

Those systems exist, but they are ad-hoc. That is, people implement their own in-house solutions as needed. We're trying to provide a language and runtime that allows developers to only focus on writing streaming applications - they don't need to worry about implementing their own infrastructure or way to describe their applications.
That makes sense. Thanks for clarifying.