Hacker News new | ask | show | jobs
by rarrrrrr 2137 days ago
Is there a reason you don't wish to batch inserts into transaction(s)? Otherwise, you're asking the database to commit (fsync to disk) between each insert statement.

Other ideas:

- Have multiple processes inserting in parallel (this interacts with the commit_delay and commit_siblings settings in postgresql.conf)

- Use unlogged tables (but table is truncated on db startup)

- Put fsync=off and synchronous_commit=off in postgresql.conf (but db may be corrupted in a crash)

- switching to BRIN indexes might be an option, but depends on data

1 comments

I do batch the inserts into a large transaction, and usually optimize for the transaction to commit in approximately ~10s. Not sure if it is optimal, but I like to monitor it like that.

BRIN indexes actually work for some of the data types I'm working with! Thank you for pointing those out!