Hacker News new | ask | show | jobs
by manmanic 2757 days ago
I've been working on DLT / enterprise blockchain technologies since 2014 and have insight into hundreds of projects, a small number of which made it to live production. Here is the bottom line: yes, it's mostly hype, but this technology does have genuine use cases - when you want to build an interparty database-driven application, and cannot find a suitable place to put the database, because of business concerns or regulation. This is fairly niche, perhaps 1% of all interorganizational database applications, but there are certainly cases where it is the right solution. The majority of blockchain projects undertaken still do not make sense, but this is gradually getting better over time.
13 comments

Can you let me know how these "private" blockchains utilize proof of X to determine consensus? E.g. proof of work, or proof of stake, or something else?

The thing I never understood with private blockchain tech is that the "traditional" blockchain (i.e. Bitcoin) relies on proof of work, and the only way this is viable is to have tons of resources working on these proofs so that you don't get a 51% attack (there have even been a bunch of articles about how smaller coins actually are very susceptible to a 51% attack by a decently funded attacker).

For a private blockchain, though, it never made sense to me as to who would serve the role of the miners with sufficient incentive to prevent a nefarious attacker. If on the other hand you are in a system where the participants agree as to how they will trust each other, well then you'd be back to a situation where the byzantine model isn't really necessary and you can just go back to a cryptographically signed ledger a la the Quantum Ledger DB that AWS just announced.

Would really appreciate someone explaining this one to me!

A useful term here is "Nakamoto consensus," which I think refers to the proof of X thing you're talking about -- consensus schemes that are resistant to sybil attacks among anonymous validators.

And I think you're right -- adding Nakamoto consensus to private blockchains makes no sense, because by definition they don't have arbitrary validators. And without Nakamoto consensus, "blockchain" is just rebranding of old and boring tech.

> rebranding of old and boring tech

A distributed merkle-chain database is still pretty innovative. A good example is git. I think what many private groups want is a "binary git for transaction data". Everybody can review their own copy of the shared chain with signing of new links in the chain.

Now that isn't really a full blockchain, but is a different kind of thing from a central RDMS.

That is a blockchain, and provides the validation and history integrity that is core to (and where the name 'blockchain') comes from.
Except the issue is that Git, which includes validation and history integrity, was invented in 2005 and predates the publication of the bitcoin whitepaper by 3 years.

The point being that all the hype about "blockchain everywhere" came about because of bitcoin. If people are now starting to refer to just the "hash of blocks with previous blocks" (i.e. the "Timestamp Server" section of the original bitcoin whitepaper) as blockchain withOUT the proof of work/stake part, well then you really are just rebranding old tech as blockchain, because it was the proof of work part that was really the new thing in bitcoin.

Can someone correct me if I'm wrong?

Proof of work is required for distributed consensus; without that you need a centralized repository like GitHub or Linus to bless the current state of the database?

There are a variety of formal consensus algorithms uses for enterprise blockchains, but they are all some variant of voting schemes based on validator signatures. Nothing like proof-of-work is needed to ensure that one bad actor, or a small number of bad actors, cannot break the network's consensus. If you have just one validator, like QLDB, then you're back to a centralized scenario.
The problem is that having a non-BFT consensus is the same that using database nodes and if you use BFT consensus you don't have the performance required for most use cases. BFT-SMaRT supposely achieve 80k tps for a few nodes but when you scale to more nodes the performance degrades substantially.

Another issue against the DLT hype is that the blockchain does not provide security where 90% of the whole architecture is outside the blockchain, less if there are many few nodes.

There might be some constant factor penalty, but we can shard BFT systems much like we do with Paxos etc. See Shasper for example -- it won't have any practical limits on throughput.
Shasper[1] "Note: This is an experimental project. Everything will break, and it may disappear without any notice!" (sic).

Until we don't see a community working on adversarial attacks, we will not know if it is secure of not. There is concrete research about BFT since 1982.

[1] https://github.com/paritytech/shasper

Is your concern that the algorithms involved in schemes like Shasper might not be correct, or that their economic assumptions about attacks might fail?

I think it's very unlikely that any of the theoretical ideas behind Shasper are incorrect. Casper itself is simple and comes with a simple proof, and it's similar to an old algorithm by DLS [1], which also comes with a proof. Sharding does introduce some other machinery, like VDFs for randomness (specifically [2]), but that has been vetted by plenty of cryptographers such as Dan Boneh's group.

So even though there aren't any large-scale deployments of BFT algorithms yet, the approach is widely thought to be sound. I'm working on a blockchain based on sharded BFT, as is Ethereum, RapidChain, NEAR Protocol, and others.

It's always possible that our economic assumptions will fail, but they're not radically different from Bitcoin's. You can attack Bitcoin by buying 51% of all hash power, or you can attack a BFT system by buying 34% of all stake. Either way it comes down to making an assumption about the attacker's funding.

[1] https://groups.csail.mit.edu/tds/papers/Lynch/jacm88.pdf

[2] https://eprint.iacr.org/2018/623.pdf

The 'corporate blockchain' described above would probably be a federated one. Participants would use private keys to sign and validate transactions, with full knowledge about who else is in the network. It's a simple quorum mechanism, or some elaboration thereof.

It is not 'trustless', but it is decentralized. You're basically relying on a majority of participants to behave ethically and not collude. In a situation where collusion is impractical or has little reward, this can function fine.

An example of this is Blockstream's 'Liquid' crypto which is validated by crypto exchanges. The security model of this coin isn't great compared to Bitcoin, but it arguably has advantages over the simple custodian model.

If the participants trust each other not to gang up on a minority but otherwise don't trust each other then BFT consensus is appropriate. As the OP said, that's around 1% of cases but maybe those cases are valuable.
HN discussion about 51% attacks: https://news.ycombinator.com/item?id=17173051
> when you want to build an interparty database-driven application, and cannot find a suitable place to put the database, because of business concerns or regulation

If you're operating a legal business, a centralized database hosted by an industry mutual or regulator beats a blockchain.

Yes, that's true. But in some cases an industry mutual doesn't exist, and the regulator doesn't want to manage the database. Then what? It can be cheaper and easier to deploy a blockchain than to build the necessary organizational structure to run a central database. Like I said, it's niche but it happens.
> in some cases an industry mutual doesn't exist, and the regulator doesn't want to manage the database. Then what?

If you're at the scale where broad co-ordination is a problem, you're at the scale where the big boys can arrange a meeting. Alternatively, if you're at a scale where someone can get everyone on a blockchain, you're at the scale to create an industry organization.

I'm sorry but this does not always apply in the real world. Someone getting everyone onto a blockchain is a one-time project, perhaps with an annual maintenance fee. Running an industry organization is an order of magnitude (or two) more expensive. Trusting someone to build an (open source) application is not the same as trusting them to centrally host it.
How about a thing that's almost-but-not-quite a blockchain? E.g. a decentralized multi-master Event Sourcing datastore. No proof-of-work, no mining, no ledger. Anyone who can connect to the network (i.e. is whitelisted) can append whatever they like, and it'll get replicated to everybody. But nobody can delete/overwrite (without losing consensus); and every event is signed by its emitter. It's just an append-only log file that happens to exist in several (geographically distant) places at once.

Get the actors together to standardize a format for the messages everyone's emitting, and then make some software (one shared implementation, many different ones, doesn't matter) to parse the log stream into a point-in-time representation you can load into an analytics tool.

Seems to me that that sort of technology would fit this use-case a lot more closely than an Actual Blockchain™ would.

> It's just an append-only log file that happens to exist in several (geographically distant) places at once.

That's the idea. Just add a proof of causality for the transactions that span more than one peer, and you've got a blockchain.

so ... sort of like git?
Every successful cryptocurrency has had protocol upgrades which require governance and I imagine private blockchains will be similar. You can't just let it run. So if you have an industry organization to perform ongoing governance of the blockchain then it is probably cheaper to have that organization run a database.
Can you please give a more specific example to illustrate the case?

From my perspective, blockchains are suitable for two things: People who want to misbehave(therefore they don't have legal recourse, independent of the morality of the extralegal actions) and people at war with each other that still need to have a relationship with each other.

It's kind of the perfect technology for a collapsed civilization or the revolutionaries and criminals in the current world order.

people at war with each other that still need to have a relationship with each other

So Wall Street banks?

No, not really :) More like Iran and Israel when they want to resolve a difficult situation in Syria where fighting is not preferred by both of the parties but they don't have a trusted 3rd party to handle the orderly resolution.
> But in some cases an industry mutual doesn't exist, and the regulator doesn't want to manage the database. Then what?

Then you have a people & process problem, not a technical one. The Blockchain, A Real Database™ or even a CSV file won't solve your problems here.

> it's niche but it happens.

Do you have an actual example?

I've only worked on a single blockchain application, but this was my experience too. I was actually pretty skeptical when I was told we'd be working on it and figured it was just a hype thing, but the use case involved a B2B process that was well-established and used a paid intermediary to correlate data across businesses. We were pretty happy with the result. I doubt we'll ever see a middleman-free decentralized utopia, but I agree that it's a technology with some genuine applications.
What was it and where? If what you're saying is true, then this is noteworthy and I'd love to look more closely at the details.
The source for the article's "Blockchain study finds 0.00% success rate" headline is "We documented 43 blockchain use-cases through internet searches ... we found no documentation or evidence of the results blockchain was purported to have achieved"[0]. That isn't a huge sample size, and "internet searches" may have skewed the results towards the more dubious projects spending large amounts on SEO.

But never-the-less I'd echo the interest in reading details of a successfully implemented use-case for public blockchain technology, beyond cryptocurrencies. Based on some of the comments here hinting that there might be some, and assuming they do indeed exist, I suspect that they are not solutions that could be achieved only with blockchain technology (like cryptocurrencies), or even solutions that are significantly better (in terms of cost, effort, time to market or some other metric) with blockchain technology, but simply solutions where blockchain has been made to work (i.e. other approaches could have been too and some of those may have been as good if not better).

[0] http://merltech.org/blockchain-for-international-development...

Sounds a little like the NoSQL trend where everybody jumped on it only to find out that NoSQL is only applicable for a few use cases.
I dont use NoSQL that much, but I also dont use relational databases much either. most of my work is in pyspark / spark on top of s3.
what?!?!? I hope that's sarcasm.
Not really. A few years ago there was a lot of hype around NoSQL and everybody wanted to jump on it only to find out that in many cases NoSQL didn't make anything better.
Yes I guess developers never really took to MongoDB, Memcache, Redis and such. Not to mention object stores like Amazon S3 which are also a kind of NoSQL database and are totally useless.
I've always seen it as the following equation: is the cost of distrusting the other parties in the transaction lower than the cost of running a decentralized ledger? if yes, you shouldn't use blockchain. this is obviously oversimplification but i think you get my point.
Yea sometimes I believe the word "enterprise blockchain" makes about as much since as a "dry rain". Basically saying that the problems that blockchain solves are usually not problems that enterprises run into often, or can't solve using some readily available alternative (e.g., SQL). Blockchain clearly satisfies a need. It's just that the great majority of enterprises clearly don't have that need.
The only successful applications of business blockchain I've seen is in businesses where nobody trusts the brokers. For example, the diamond business has to keep track of where the diamonds came from and nobody trusts the brokers to not lie, so blockchain works here.
How does the blockchain keep brokers from lying?
Maybe you trust the source, but not the broker?
Exactly - if the diamond mines are honest then you don't really need to trust the middlemen. You could also see how many diamonds a certain mine was claiming to produce, and if that number seemed disproportionate to the size of the mine, people might stop treating diamonds coming from that mine.
I've never encountered a system where an un-trusted middleman didn't get out-competed, driven out by the buyers or (in one case) actually killed.

Hasidim don't seem like they have that problem.

Wasn’t this type of stuff around well before the hype of the blockchain? E.g distributed hash tables?
when you want to build an interparty database-driven application, and cannot find a suitable place to put the database, because of business concerns or regulation

Technology can't solve this problem.

If multiple entities need to coordinate inside a mutually used product don't trust each other, then the problem is structural and needs to be solved with interpersonal, regulatory or political action.

Annnnd... if you have a structural problem which cannot be corrected via interpersonal, regulatory, or political action, then what do you do?
Quit and start over, or move. That's why people migrate between countries or quit jobs.
Ah, if only it were that simple.
No need to be glib. The kinds of situations that are being discussed as the best for blockchain are literally the most intractable - hence why I said it was not a technology solution.

If you're in a situation where cryptocurrency is the best solution for currency, then you're probably in a lawless wasteland with abjectly destructive governance systems.

Seems like in that circumstance moving is the right answer.

I'm not being glib, I'm observing that the global system of passports and visa controls makes "moving" nearly impossible. I nearly pulled it off just over twenty years ago, but I failed, and now I'm permanently stuck back "home". Cryptocurrencies won't help, they're just another way to do capitalism.
Yep, i've been trying to explain this to people on here for quite a while now. This is correct. Blockchains allow data and co-operation to be domiciled nowhere, which allows competitive actors to agree and co-operate.

It also enables otherwise untrusted actors to make trustable claims, bypassing the gatekeepers that would otherwise have mediated those claims. One tangible example is ICOs bypassing not just regulators, but banks and the financial industry that would normally have had to underwrite them. Obviously this particular example has some serious kinks to work out, but the ability to bypass the corporate gatekeepers (not so much the regulators) has value, I think.

So can you help the authors of this article with their quest to find any success stories at all?
> This is fairly niche, perhaps 1% of all interorganizational database applications, but there are certainly cases where it is the right solution.

So I personally think DLT is going to be one of those multi-quadrillion dollar technologies that come around every few hundred years. The real benefit of DLT is that it enables new types of human relationships. So thinking about it in terms of what percentage of your existing databases should be replaced by it isn't going to give an especially impressive result, because by definition your existing databases are going to model your existing relationships.

Think about what percentage of the stuff in your house you would own without the invention of double entry accounting. Unless you happen to have some veggies from the farmer's market or a sweater one of your relatives knit you, the answer is probably 0.00%. It simply isn't possible to manufacture things like the iPhone without double entry accounting, because without double entry it would be impossible to form the sorts of human relationships needed to produce such a complex product. And if we asked someone to take a look at all the stuff in their house a couple hundred years from now, I'm guessing that a similar 0.00 percentage of the stuff they own will have not have been created as the result of DLT.

The fact is that once DLT becomes ubiquitous it will no longer be cost competitive to manufacture and distribute products using only the sorts of relationships and techniques enabled by double entry accounting. And if anyone even tries it's going to be like bringing a knife to a gunfight.

I can’t tell if you are being sarcastic or not, so I assume this is a serious comment.

Can you share some examples of use cases where DLT is going to be transformative?

I mean a good example of the need for DLT is the fact that no one in the country can buy romaine lettuce right now. Does it really make any sense whatsoever that no one in the country can eat lettuce for the next couple months just because one farm got contaminated with E. coli?
You seem to be suggesting that a distributed ledger would have made it possible for regulators or buyers to track the specific source of the contamination and pull it out of the food system. DLTs are starting to be used for that level of tracking for diamonds, but key to that system is the ability to establish a "fingerprint" for each diamond that involves a laser-inscribed serial number on each diamond.

So to apply that, wouldn't we have to somehow register each individual head of lettuce with a tamperproof fingerprint? Just registering the bags the lettuce came in isn't good enough; the "farm" selling you their romaine might actually be selling you romaine from somewhere else that they've repackaged. (If that wasn't the case, we wouldn't be having this lettuce problem to start with.) And this is setting aside the possibility that lettuce might be leaved/shredded before packing and shipping, because now we have to put that tamperproof fingerprint on each and every leaf.

Oh, also there's the thing about all the leaves actually being in contact with one another as they're shipped, so by the time the consumer actually eats the leaf and gets sick, the best you could possibly do is say "we think it's from one of these packagers."

Maybe you're envisioning some other way entirely for Distributed Lettuce Technology* to solve this problem, I dunno. But I have trouble seeing it.

*I'm sorry, but cut me some slack, the joke is RIGHT THERE

> the "farm" selling you their romaine might actually be selling you romaine from somewhere else that they've repackaged.

Think about it from a game theory perspective. Under the status quo, each party in the supply chain maximizes their profit by lying about where their supplies came from. Whereas with DLT, each party maximizes their profit by being honest about where their supplies came from.

As an example, let's say you hijack you a truck and swap out expensive lettuce for cheap lettuce in a way that circumvents whatever tamper proofing technology has been applied. Stuff like this happens all the time currently, and is wildly profitable. (That's why if you go to a sushi restaurant and get a bunch of assorted sushi pieces or rolls, there is roughly a 0% chance that your meal will actually consist of what you ordered.)

With DLT though this is no longer profitable, because if you sell the cheap lettuce as expensive lettuce then you now have to sell the expensive lettuce as cheap lettuce, because each head or crate or whatever still needs to be traceable back to the source. This means that your total profit from the transaction is just whatever you would have made without cheating, plus your costs of hijacking the truck. So all in all, a substantial net loss.

Doesn't this require you to trust the source? And if you trust the source, there is no need for a blockchain?
Could you talk me through how a blockchain solution would help avoid this in a bit more detail?

I've seen lots of vague "blockchain for supply chains is a great idea" pronouncements but I don't get how it would actually work.

Sure, so each party in the supply chain registers their private key with a PKI by going to the PKI's website (e.g. GoDaddy) and sticking their USB fob into their computer. The PKI associates the private key with an identity, and allows private keys to be voided and replaced if they get lost or whatever.

Then as each head of lettuce is picked, it goes into a crate that's securely sealed and then signed with the farmer's private key in a way that originates that crate of produce on the ledger. When the farmer sells their produce to the middleman that transaction is also recorded, and so on, all the way to the end consumer who buys the produce in a grocery store or in a restaurant. (And consumers would just use their phones or credit cards for this, rather than using any sort of external fob.)

Then when the first person gets sick they report their illness as per usual. Nothing happens at this stage, because there's no way to narrow down what made the person sick. By by the time the second person gets sick (with E. coli of the same genetic signature), now you can find the furthest place back in the supply chain where both people's purchases intercept. So you can now see if e.g. the contamination came from a single farm, and if so only recall lettuce from that farm rather than all romaine lettuce produced worldwide.

Because the database is open anyone can download a copy, and there is no risk of a single entity imposing a 30% Apple tax on each head of lettuce or whatever. And each person benefits from participating, because it's a pareto improvement in terms of their profitability. (Now their products only get recalled when they are at fault, rather than their products getting recalled when anyone is at fault.)

How about the types of products that can't be stored in a secure package from producer to consumer through the supply chain but need somehow be processed within the supply chain? You know, at least something like 99.9999% of the products...
Suppliers at every step of the chain enter data into a blockchain. Every organization runs a node. Therefore they can’t lie later and tamper with records when something goes wrong. Investigators can trace provenance easier. That’s it really. An append-only cryptographically secure database would do that same thing, but that’s just another name for a blockchain, which is a rebranding of a specific type of distributed database that has enhanced trust properties.
Blockchain requires distribution for trustless implementation though, which is the duplicated expense (versus an append-only secure database).
Putting lies into a blockchain doesn't make them true. Lettuce can still kill you in a post-blockchain society.
No, but it isn’t about lying as a matter of business, that would easily be detected and caught in most cases, and a large amount of work to continually lie effectively. If you have employees and machines writing lots of data it’s infeasible to tamper with it in real-time.

The tampering comes after the fact when a legal issue arises. The immutable nature of the ledger makes it an improvement over the past by preventing later tampering.

So why not use a central database and allow all stakeholders to replicate and keep their own logs of edits. It would be far more efficient. It would lack a certain buzzword though.