[sghemawat]

One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects.

However because of a fundamental difference in data structures (TokyoCabinet uses btrees for ordered storage; leveldb uses log structured merge trees), random write performance (which is important for our needs) is significantly better in leveldb. This part we did measure. IIRC, we could fill TokyoCabinet with a million 100-byte writes in less than two seconds if writing sequentially, but the time ballooned to ~2000 seconds if we wrote randomly. The corresponding slowdown for leveldb is from ~1.5 seconds (sequential) to ~2.5 seconds (random).

[lisper]

Did you consider fractal trees?

http://tokutek.com/presentations/bender-Scalperf-9-09.pdf

[yingfeng]

fractal trees should have a faster read performance with leveldb with a comparable updating performance. There does not exist any open source solution for such structure, while we could hope that www.acunu.com might provide an open source solution for similary Stratified B-tree http://www.acunu.com/2011/04/log-file-systems-and-ssds-made-...

[br1]

Being in-kernel, acunu is very far from the lightness of LevelDB.

[psaccounts]

Is there a more useful technical paper on Fractal Trees? Better yet is there a open source implementation of the same?

[jbapple]

Cache-oblivious streaming B-trees:

http://supertech.csail.mit.edu/papers/sbtree.pdf

I don't know for certain any open-source implementation, but I have heard COLAs are used in HBase.

[davidhollander]

> difference in data structures

Always good to have more tools in one's arsenal in that case. I'll drop one of you a message if I write a Lua binding for it (using LuaJIT+embedded k/v for data services atm).

[spullara]

Have you looked at implementing CAS? It is the one thing missing for me to adopt it as a underlying store for my AvroBase API: http://www.javarants.com/2010/06/30/havrobase-a-searchable-e...

[cmartella]

Could you please report about caching setup for random read test? was there any cache (a part of OS's) for either position/index lookup or payload? Or did you hit the disk without any caching of past operations?

thanks