Hacker News new | ask | show | jobs
by grafraf 686 days ago
We are storing the result of the parsed scrape as parquet. I would advice to store the raw data as well in a different s3. The database should only have the data it needs and not act as a storage.