Hacker News new | ask | show | jobs
by amluto 301 days ago
> This is a lot of fuss when you can get a batch update to stay within a few minutes of latency.

Replying again to add: cost. Just because you can do a batch update every few minutes by doing a full scan of the primary key column of your Iceberg table and joining against your list of modified or deleted primary keys does not mean you should. That table scan costs actual money if the Iceberg table is hosted somewhere like AWS or uses a provider like Databricks, and running a full column scan every three minutes could be quite pricey.