[jandrewrogers]

A point that a lot of nominal Big Data startups miss is that genuinely large-scale data management and analytics are not driven by visualizations at all nor fit in a web-driven SaaS-like environment. The purpose is to answer a complex question from unimaginably large volumes of data, not to draw charts and graphs. It is often too I/O intensive for virtualized clouds and the visualization component is almost superfluous to the purpose. Most of the problems that need to be solved in Big Data are low level, down at the computer science and infrastructure level. Many of the use cases are intrinsically poorly suited for web-based SaaS type offering.

To make matters worse, many high-value Big Data analytical problems are (literally) not meaningfully visualizable except for marketing purposes. It is rather tricky to visualize an analytic product when there are a hundred critical values that need to be rendered in some fashion for every pixel your monitor can display. A lot of high-value analytics have this characteristic but most of the nominal Big Data visualization tools ignore this case even though it is arguably the most important one.

Consequently, while labeling your startup "Big Data" is trendy and fashionable, there are very few genuine Big Data startups. Adding value in this market requires a combination of serious theoretical computer science chops plus very creative interface design. Few startups are actually addressing the needs of this market and are instead assuming the market wants the web app they have the skills to produce.

[mwetzler]

Agree! My personal experience: our customers at Keen IO constantly demand better visualizations and many expect a traditional analytics frontend like GA. However... our highest value customers pay for our API capabilities, not our line charts. Storage and querying at massive scale is the hard part and that is worth thousands of dollars a month, not tens or hundreds.

Stunning visualizations and a better web experience are definitely something we want to do when timing allows, but so far our true customer value is in the backend and our APIs.

[Mercutionario]

This is presented in a much harsher light, but you're exactly telling the story the way it happened for us.

We got completely blindsinded by the idea of a "big data product" and we quite naively replaced "product" by "trendy webapp with shiny charts" in our minds, whereas there is definitely the place for a technological product adressing those needs, but those are deep tech product paired with services because they require setup & maintenance etc... For instance our friends at infochimps do it really really well, and they're on a good path to a big data product.

[mathattack]

I hear you on visualization. A lot of big data is just, "I have a billion row, hundred column table that I want to join with 5 or 6 other tables, and do some time series math." It's computationally intense linking of tables, but not necessarily visualization.