Hacker News new | ask | show | jobs
by fabian2k 741 days ago
I think the clickbait title might harm the reception here somewhat. This kind of solution is potentially interesting, but it's a solution with many tradeoffs. Keeping a copy of the data synced has a cost in space and in general resource usage. And of course the analytical queries will work on slightly delayed data, which might or might not be an issue for your use case.

The part I'd wonder about is how expensive this is after the first sync, especially for a table with many writes. And related to that how long the delay can get between your live data and the column store version.

1 comments

Hey Fabian, there are indeed many trade-offs in making this possible.

The extension periodically schedules export runs for a table exporting all the data each time (I know this is expensive but I haven't found an alternative like a change pump for getting writes after time T in Postgres yet). The export frequency can be configured by the user.

Regarding how much the delay will be, we that depends on the export frequency and the machine running the Postgres instance. A fast machine could complete an entire table export every hour or so resulting data that is decently fresh. Slower / more resource constrained machines would have to schedule exports at a slower frequency.