[mighty_warrior]

This sounds a little over complicated all in an effort to decrease the amount of code changed at any given time. They took great pains to keep data in sync across A and B datastores and I'm not so sure that extra cost was worth the perceived stability of this approach.

[perlgeek]

> They took great pains to keep data in sync across A and B datastores and I'm not so sure that extra cost was worth the perceived stability of this approach.

Such great pains come with huge systems. What's the alternative?

Taking the platform offline for a few hours? Management will say no. Or maybe Management will say yes once every three years, severely limiting your ability to refactor.

Doing a quick copy, and hope nobody complains about inconsistencies? Their reputation would suffer severely.

[mighty_warrior]

They maintained a replication process across both tables as they updated the read processes before updating the write process. Say for whatever reason their offline replication process broke for 2 hours. For those 2 hours of downtime that replication is broken, the system is reading from a table that is not in sync with the table that is receiving writes. At that point you are displaying incorrect subscription data to your customers.

[sorkin2]

> They maintained a replication process across both tables as they updated the read processes before updating the write process. Say for whatever reason their offline replication process broke for 2 hours.

From the article I got the impression that both tables were being written to in the same database transaction, so this is not a possible failure scenario at all.

[ceejayoz]

Stability matters when you're dealing with people's bank balances and billions of dollars a year of credit card transactions.

[smnc]

There's only one datastore. I'm not sure what you mean with "perceived stability", it's about data integrity and preventing data loss during a model reshuffle.

I'd like to hear about alternatives if you've had experience.

[mighty_warrior]

They provided data integrity with a background sync process. What would have happened had that sync process failed?

In the past I have either flagged records to say where the system should read the data from, or just built logic into the readers to say if there is no data in A, read from B.

Then you update the writers to migrate the data from A to B on every new update and remove the data from A. It is an expensive 1 time write to move the data, but then you don't have to worry about keeping data in sync across two storage locations.

What you end up with is all records that are actively getting worked on move first. At a later date you start migrating all those stale records from A to B with a background process. Once A is empty, remove the logic to read from A, remove the migration writes, remove the A datasource.