|
|
|
|
|
|
by lixtra
2857 days ago
|
|
|
> It is similar to Berkeley DB, LevelDB or SQLite4 LSM. Can you point out the pros and cons vs above projects? > reading from transaction while resetting it can lead to unpredicatable consequences. I read this as: only use it single threaded. |
|
|
Briefly speaking, the above projects are way more mature, full-featured and use different data structures for storing data. BDB, LMDB are B-tree-based, LevelDB (and forks), SQLite4 LSM are using log-structured merge-trees.
TKVDB is a lot simpler, a bit more low-level, uses Radix trees and not tested as well as other key-value engines.
> I read this as: only use it single threaded.
If you protect access to data with your OS locks, everything will be fine.
We're using TKVDB in multi threaded environment (in RAM-only mode) for IP lookups with Netmap and DPDK. On 40Gbe full line rate there is only few hundred CPU cycles for decision on network packet, so the locking tricks becomes important.
Intel has guarantees about writing properly aligned data - if one core is writing 1, 2, 4 or 8 bytes, it will be written atomically. We're exploiting this feature when updating our tree - final update operation is 8 byte (4 byte for 32 bit CPU's) aligned pointer update.
So, in some cases you don't need to use locks at all (you may add HW memory barrier (__sync_synchronize()) after tree update, but it will slow down writer). If only one core is writing data, another cores can safely (well, almost) read it. Warning in docs was about this lockless case.