Hacker News new | ask | show | jobs
by rch 4160 days ago
I was thinking more like 200M rows and 16M columns (sparse - stored as compressed rows), or perhaps a grid cell for every geohash address. The only solution I know of is part of custom GUI toolkit rendered with LWJGL in a desktop app that handled the former case (crashed the first time - worked fine a week later). Obviously far fewer elements would actually be resident in the client at any given time.

More common is that I just don't know how many columns there will be (or the datatype, optimal width, etc) until the user performs a series of actions and renders a particular view. I've shoehorned a solution into extjs that works (not scrolling, just dynamically defined), but I'm always looking for options.

1 comments

I was thinking more like 200M rows and 16M columns

I don't think a user would be able to meaningfully navigate a table like that by scrolling it. How would they cognitively track where they are or how to return to interesting parts of the data? IMHO, you should rethink how to display and navigate this data. The user would need sophisticated search/filtering support.

It's the same as zooming in and out in a google map - if you zoom out, the server is providing e.g. averaged data. Thinking about trackpad navigation around a map of tiles (sections of a spreadsheet) is probably more relevant than scrolling per se.

A side window let's you navigate at a high level (low res) and zoom in to data-dense areas. Low level / high res navigation in the grid might be used for exploring data from individual sensors, in context with interpolated values.

You can keep the scrollbars from becoming irrelevant by scaling to the size of a logical grid 'neighborhood' of adjacent tiles at the current zoom level.