Hacker News new | ask | show | jobs
by ngrilly 3223 days ago
According to PostgreSQL documentation, "archived files that are archived early due to a forced switch are still the same length as completely full files".

I'm afraid to use a lot of storage for WAL segments that are mostly empty:

16 MB per segment x 60 minutes x 24 hours x 7 days = 161 GB/week

Does WAL-G/WAL-E compression help?

1 comments

Yes, the lzop compression helps a lot, and I imagine that mostly "empty files" will be more compressible.

On a staging server with little activity, the compressed WAL-E wal files go as low as 1.9MB per 10 minutes. (~2GB/week)

The production server has files between 4 and 12MB per 1 minute or less. (~220GB/week)

WAL-E has a good `wal-e delete retain` command that removes older base backups and wal files.

2 GB/week is so much better than 161 GB/week! It looks like compression helps a lot. Thanks for sharing these numbers.