Hacker News new | ask | show | jobs
by tracker1 285 days ago
Cool... though that would make it difficult to get the hundred or so CSVs into a single table, since it isn't supported I guess stitching them before processing would be easy enough... also, no idea when that feature became available.
1 comments
JustExAWS 285 days ago
It’s never been a good idea to batch ingest a lot of little single files using any ETL process on AWS, whether it be DDB, Aurora MySQL/Postgres using “load data from S3…”, Redshift batch import from S3, or just using Athena (yeah I’ve done all of them).
link
tracker1 284 days ago
These weren't "little" single files... just separated by tld iirc.
link