[matc]

We are actually working on this at ChronicDB http://chronicdb.com

On 6ren's rigidity argument, relations and SQL are still not enough to guarantee you'll always be able to get the data you need; notably, when the data model changes. Backward compatibility is important and we might have they only solution that guarantees it.

[6ren]

> relations and SQL are still not enough to guarantee you'll always be able to get the data you need; notably, when the data model changes

What data model changes are you thinking of? Relations and SQL will cope with reorganizations/restructurings of the data, provided the information content is the same. But won't cope if the information is different (e.g. info is deleted or added).

> "Only ChronicDB can run new applications and unmodified old applications against a master database without breaking anything." [from your link]

I like your points about undoable DELETE (do modern RDB's really lack that?); but the above quote is striking because it was the stated motivation for relational databases in Codd's 1970 paper: http://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf

[from abstract] Activities of users at terminals and most application programs should remain unaffected when the internal representation of data is changed and even when some aspects of the external representation are changed.

[from conclusion] In Section 1 a relational model of data is proposed as a basis for protecting users of formatted data systems from the potentially disruptive changes in data representation caused by growth in the data bank and changes in traffic.

The paper discusses derivability, and how some relations (such as those needed by old applications) can be derived from named relations using a high level language (now SQL). The idea was that you can change the database schema yet still provide the old relations (this mechanism is now called "views").

I'm not saying ChronicDB has no role, just that this one particular aspect, in isolation, doesn't seem new. But if it facilitates back-compatibility in contexts where RDB doesn't work well, it could be a big win. Back-compatibility, like integration, is always in demand.

EDIT looking at the "schema migration" example (http://chronicdb.com/system/files/images/agility_full_snap_g...), it looks like a join, with renaming, and a type change... is the new thing offered in this example that updates are bidirectional (especially for the type change)? Or is it that both old and new relations have the same name, "customers"?

[matc]

You are right, this aspect is not new. More like half-baked.

I'm not sure what you mean by bidirectional. If this suggests having a separate copy for the old data and the new data then, in ChronicDB at least, no they are not bidirectional. Which is what one wants ideally: data consistency.

If you mean that updates on the old version are immediately reflected in the new version, then yes.

Basically, there's a single version of the truth, while old and new relations can have different names, for all DML operations.

[6ren]

Yes, the second one. So that from the old app's point of view, nothing has changed: it can create, read, update, delete as before. A single version of the truth etc.

SQL's "CREATE VIEW" does this already (note: not all views can support CRUD, e.g. if you changed an aggregate column, like COUNT, there is no clear meaning for the affect on the underlying tables. If you increased count, what rows should it add? If you decreased count, which ones should it delete? The problem with aggregates is that the mapping from the underlying tables to the view isn't 1:1, you can't run the mapping backwards to reconstruct the original).

1. How is SQL's solution "half-baked"?

2. What does ChronicDB add?

I mean, back-compatibility is valuable, but what is the problem you see with the existing solution?

[matc]

1. Thank you for the link on views being updateable with some (natural) restrictions (http://news.ycombinator.com/item?id=3406952). Given this feature, SQL offers the primitives needed. Even if this feature is missing from various databases at the moment, it should be available eventually.

But updateable views are still not what you want! See below.

2. What ChronicDB adds in terms of backward compatibility is, effectively, automatic creation and management of these views for every schema change.

Rather than manually write compatibility views and rewrite/review an application to ensure it always uses these views, programmers instead access plain tables, not views. In fact ChronicDB creates no views at all!

You want to issue SQL queries that don't break. You don't want to manage views to do so. It's a subtle, yet major point. No database to date offers such automated versioning.

ChronicDB of course adds other features we wanted to have from our database (reversibility, data versioning, zero-downtime relocation) again in the spirit of saving the programmer/DBA from doing anything manually.

[6ren]

2. Yes, I agree, nice to have the "view" created automatically when you change schema.

Codd's idea was also that applications needn't be aware of the change (a view appears as a plain table)... but (in contrast to your approach) the views were managed by the DBA who was making the schema changes. So I think this feature was present from the very beginning, being the initial motivation for relational databases. [I looked up MySQL because that's the DB I have access to, but it's a general feature: http://en.wikipedia.org/wiki/View_(database)#Read-only_vs._u... ]

I would expect automatic versioning to be available too, perhaps only in the older and more expensive RDBs. I've come across quite a few research papers on schema evolution in relational databases - some from big RDB vendors. I don't recall any on automated versioning, but I wasn't looking for that. So this is just my guess.

However, if you're not seeing databases with automated versioning, then (even if some highend RDBs do have it) there will be customers who also aren't seeing it (or it's not applicable e.g. it's too expensive). A market for you. And if you can make it truly painless (or even just less painful), that's really something.

I don't see any problem with your other features. The one feature of mapping for back-compatibility is of interest to me, which lead me to study Codd's original paper in great detail over a few weeks. That's why I've been hassling you on this issue.