Hacker News new | ask | show | jobs
by 127001brewer 3878 days ago
This might be too much of a newbie question, but how do these types of data stores (such as "data lakes") get populated?

For example, and again I might be too uninformed here to ask the correct question, do you use ETL-type tools to get and store data? Or is it usually scripts that pull in (and process) data from various sources?

1 comments

With ETL, usually you start out with handmade scripts, and then depending on how rich you are and/or how complex your situation gets, you possibly move up to either an internal tool or a vendor tool.