Hacker News new | ask | show | jobs
by trentmc 3146 days ago
Hi, it's Trent here, CTO at BigchainDB.

Summary: Python isn't the bottleneck yet, and if it becomes one, C will become the last 1%.

I've been working on production apps in Python since 2002, including ones doing large-scale compute running 1000+ machines at once. How: 99% python, 1% C. But the trick is, you only build in the C once you've worked out all the kinks and optimized the big-picture stuff elsewhere. Python is great for not only connecting things, but rapidly iterating on algorithms and building maintainable code.

The AI / ML community has discovered this too: Python is now the most popular language in that community. Despite the heavy compute. How: most of the popular libraries have efficient C (etc) implementations under the hood.

This is exactly the philosophy we've been following at BigchainDB, with success. Python to connect things, iterate quickly in improving algorithms, and ship maintainable code. We haven't got far enough to resort to building our own C libraries yet, though many 3rd party libraries we use are implemented in C.

[EDIT] Based on the comments below, I'll now mention here too: BigchainDB wraps MongoDB, which is written in C++. And, Python 3.5+ (which BDB uses) has gradual typing, which brings many benefits of static typing to Python.

5 comments

Is there a successful database written in python? I probably need to dig through your code a bit more to understand how your utilizing it.
The actual database is not written in python.
Good point - BigchainDB wraps MongoDB, which is in C++. I updated my comment above to help future readers.
Trent, dean here. met you at ICCAD few years ago (2014?) and we had dinner with Bill Swartz at an Asian place. We discussed bitcoin and I remember it was $600 back then. Funny that it's now 6k$ and I didn't know you're gonna be in this space. Just wanna shout out that bigchaindb looks legit/cool and good luck in the future!!
Oh cool! Thanks for the ping, Dean. I hope you're well. Yeah blockchain space is a bit crazier than EDA, but some of us are engineers bringing our discipline and experience to this new field:)

I encourage you to give BigchainDB a whirl, it's easy: https://www.bigchaindb.com/getstarted/

Will definitely kick the tires on this one. And people should take notice that you guys didn't raise funds through an ICO. If anything, that right there is a big sign that Bigchain db is a cut above the rest in terms of riding the block chain hype train.gl
+1 :)
Another hi from the past! I'm going to have to check this out, since, well, it's you and there's been some discussion about "blockchain applications that aren't just trading currency" at work. Cheers! Glad to see you're still working on crazy interesting shit!
Hey Tony, great to hear from you! Yes please do check this out, we think it's interesting too:) Also interesting is BigchainDB's sister project www.oceanprotocol.com. Cheers!
If this db is blockchain-like, does that mean that every node must carry a full copy of the db?
To me, "blockchain-like" is about getting the benefits of decentralized, immutability (tamper-resistance), and assets.

How one goes about getting those characteristics is wide open. Most blockchain systems do have a full copy of the database at each server node, i.e. fully replicated. Also, they are "peer to peer" which means there is no distinction between clients and servers. (They do have SPV wallets though which is kinda similar.)

BigchainDB's focus has always been about scale. We're partly there but not fully: we are currently fully replicated but are targeting sharding to address that [1]. Where we do get scale already is properly distinguishing between between clients and servers. Servers are "super peers", decentralized among themselves. They do the heavy lifting, i.e storage. Apps don't need to run a server node; instead they simply are clients to the network, and of course can query >1 node.

[1] https://blog.bigchaindb.com/bigchaindb-developer-update-2d32...

That still doesn't encompass the advantages of a statically-typed language...
I totally acknowledge that there are pros and cons for both dynamically and statically typed languages. (We could start a 10 page discussion here. But I've been there before, perhaps you have too, how about we save our energy? ;)
For the latest and greatest 10 pages about that; just search for the the Hickey 10 year Clojure (rant-ish in places) talk and the Haskell community responding to it.
OK! :)
Well done for gracefully and thoroughly responding to the usual HN barrage of technical critiques.
Thank you:) Even if there's hype and critique, we can always rise above and maintain our integrity as we build things that matter.
Python has optional typing support in the standard library since version 3.5. Granted, not type _checking_, but that is offloaded to external tools like mypy[0].

I feel that gradual typing offers the best of both worlds: rapid prototyping, but also the strictness of types where you need it.

0: http://mypy-lang.org/

Good point! I think the Python folks made a great design choice there. I updated my comment above to help future readers.