[nickpsecurity]

"enabling TAPIR to provide the same transaction model and consistency guarantees as existing systems, like Spanner, with better latency and throughput."

Oh, hell yeah! Now that's great stuff. Can't wait to see the next step done by this or another team: building an alternative to the F1 RDBMS that Google built on Spanner. Would give CochroachDB some competition.

[msackman]

I'm the author of GoshawkDB.

I've just been watching the talk on this at https://www.youtube.com/watch?v=yE3eMxYJDiE. GoshawkDB has a very similar design wrt the messaging and replication design. In fact, in some places, it appears that GoshawkDB's design is a little simpler.

There are obviously many differences too: for example GoshawkDB runs transactions on the client, GoshawkDB uses Paxos Synod instead of 2PC, and GoshawkDB clients only connect to one server so there are 2 extra hops, but that's a constant so from a scaling pov, it should behave the same.

One of the biggest differences is GoshawkDB uses Vector Clocks (that can grow and shrink) rather than loosely synchronized clocks.

This TAPIR work does look great - I had no idea that it was ongoing. I'll read through the paper too, but it's great that GoshawkDB has so many design ideas in common.

[nickpsecurity]

Very interesting. I went to search for your stuff only to find I had this bookmark:

https://news.ycombinator.com/item?id=10778946

So, I knew it was interesting and possibly great work but didn't have any time to look at it. I'll move it up the backlog a bit. Maybe at least give the papers and stuff a good skimming tonight. :)

Note: Parallel, ongoing, similar work going unnoticed is a huge problem in both IT and INFOSEC. I have a meme here about how we constantly reinvent the same solutions, make the same mistakes, or move at a slower pace due to lack of connections with similar work. I keep proposing something that's like a free and open combo of ACM or IEEE with members who are academics, FOSS coders, pro's... people that contribute at least in writing. Stash papers, code, and forums there. So, odds of serendipity increase. Thoughts?

[msackman]

Heh, definitely! I could ramble for hours about how there are so many disincentives to share anything until launch - the whole industry really isn't set up to actually make progress. However, multiple independent co-invention of stuff is still valuable from a validation point of view. The worst bits I see are when something from 20 years ago gets reinvented. I have emailed Irene just to see if there's anything or anyway to collaborate.

[nickpsecurity]

Great! It's awesome to see you taking the initiative! Might be an exception in the making.

Im about to leave for work but quick question. What I want to see is an open equivalent of Google's F1 RDBMS: the best one. Does yours already provide its attributes, is it more like Spanner jnstead, or what? Aside from CochroachDB, where is OSS on a F1 competitor?

[msackman]

So GoshawkDB doesn't have an SQL engine currently, so in that way it's probably not comparable with F1. GoshawkDB stores and accesses an object graph. Hopefully the howtos on the website will help people get into the mindset.

I'm not sure if it's worth trying to compare anything to Spanner or F1 because unless I'm mistaken, no one outside of Google can use F1 or Spanner - they're not released. So who knows what the performance of these things actually is? There's no way to verify the claims made in any Google paper.

[nickpsecurity]

"So GoshawkDB doesn't have an SQL engine currently, so in that way it's probably not comparable with F1. GoshawkDB stores and accesses an object graph. Hopefully the howtos on the website will help people get into the mindset."

Gotcha.

"I'm not sure if it's worth trying to compare anything to Spanner or F1 because unless I'm mistaken, no one outside of Google can use F1 or Spanner - they're not released. So who knows what the performance of these things actually is? There's no way to verify the claims made in any Google paper."

I think it's worthwhile for these reasons:

1. They describe it in enough detail in their papers for comparisons or maybe clones to be made.

2. That's led to competitors or open-source clones of tech before. Remember map reduce?

3. They've deployed it operationally.

4. Since when can Google not handle a backend tech it claims to be able to do? I thought their rep was solid on that stuff.

So, Google already as a strongly-consistent, scalable, multi-datacenter RDBMS with NoSQL-like performance. If good on enough workloads, that's the best thing I've heard in that category since NonStop SQL. The GPS thing, while brilliant hack, might be hard for adoption. An improvement area CochroachDB people are already targeting. A full clone or competitor could rock DB world as a great alternative to Oracle clusters or NonStop for mission-critical workloads where some delays were tolerable.