Hacker News new | ask | show | jobs
by mgkimsal 3070 days ago
and I think that's what what johnny555 is questioning - isn't this common knowledge already? apparently not.

here's another tip, at least on mysql, but possibly other databases that have memory tables. Import stuff in to memory tables, then insert from the memory table to a disk-based table. I took a process that was naively importing data via SQL commands which took close to 24 hours down to around 20 minutes by breaking it up, chunking imports to memory tables, then copying those to permanent disk. This was years ago (12?) and mysql is probably better about insert handling than it was, but that approach (plus the drop/recreate indexes) meant this was a smallish process vs a 24 hour import cycle.
1 comments
mbrumlow 3070 days ago
> apparently not.

Just becasue one blog post making it seem that way makes it not known ?

link
mgkimsal 3070 days ago
not just because of that - cuchoi also seemed to imply that.

relatedly/anecdotally, I still run in to people who aren't aware of this, as they don't understand what happens when you're inserting data in to a database or what indexes are in the first place.

link