Hacker News new | ask | show | jobs
How Fast Can We Write a 10M Row CSV to a Postgres Database with Pandas? (innerjoin.bit.io)
3 points by otacust 1571 days ago
1 comments

I wrote this article after a colleague pointed out that the Pandas DataFrame.to_sql() method uses row-by-row INSERTs. There are plenty of good reasons for this, and the to_sql method works great with many different SQL database flavors, but it's not fast.

This article compares the performance of different methods for writing a Pandas DataFrame to a PostgreSQL database using the to_sql method on DataFrames ranging from 100 rows to 10,000,000 rows.