Hacker News new | ask | show | jobs
by LambdaAlmighty 4550 days ago
Lost me at "big data problems".
2 comments

Big Data has so many meanings. It really is a little absurd but I guess that is the life of a buzzword. My take on Big Data is 2 parts 1) Big Data is managing the large transactional, daily data of an organization. I believe this is what most people relate to Big Data i.e. map reduce, BI, etc. 2) Big Data is gathering bits of perceived dissimilar minutiae and finding meaningful information from it. This is classical statistics, engineering, and machine learning. The data itself may not be the scale of Amazon, Google or Facebook but it still computational intensive to develop algorithms for it.
Why? A multi giga dataset in machine learning is considered to be big enough to be called like that. An algorithm like support vector machine which complexity can be O(n^3) would be prohibitive in a single computer.