|
|
|
|
|
|
by devnull791101
3564 days ago
|
|
|
a good place for NoSql is where you don't know what data you are going to receive. e.g. i get weather data from a number of different sources in a number of different formats. the sources and formats change fairly regularly. i am able to extract the data in key/value pairs but don't know which keys will be in any given set of data. with nosql you can extract any key/value pair and save it to the database without needing to parse the data |
|
|
That can be good strategy when dealing with changing data formats:
1) Import the data "as is" into the database.
2) Process the data, extracting the data you need for querying.
3) Whenever parsing fails, figure out why and fix it (update your source code, throw away bad data, whatever), then rerun the 'process' step for that data.
Reparsing can be made reliable if you use transactions in the step 2. With a NoSQL solution, it may be hard to guarantee that you don't lose a few records ('may' depends on the specific NoSQL solution and the amount of manual work you are willing to do to restart your pipeline). (You can also postpone discovering your data problems, but that's delaying the inevitable)
What NoSQL is particularly good at is running on multiple machines. It achieves that by giving up some letters of ACID.
It also typically makes it easier to store and query unstructured data than SQL databases do, but SQL databases are catching up.