[kgeist]

Sounds neat! We currently have to deploy 2-3 releases to just have 1 logical migration and it's pretty error-prone and complicates our release schedule. We basically gave up trying to rename columns at this point because it's too much effort for little worth, but it confuses the hell out of new devs when a column name in a DB doesn't match the name in the code.

Your article mentions that it allows to avoid locking tables during migrations and it's something we haven't solved yet either - in one of the recent migrations that touched a very fat table we had to write a custom migration which applied changes in batches to avoid locking the entire table under one fat transaction.

Our DB is sharded: we first apply migrations on thousands of DBs and only then deploy code. Such migrations can take up to an hour to finish before code is deployed, so we have to make sure our migrations are forward-compatible because old code can see new schemas for a prolonged time. Will Reshape work well for this mechanism? Sometimes we have to run custom migrations which call code, I guess Reshape can't replace it because it's strictly DB-based?

Unfortunately, we use MySQL, not PostreSQL, so I wonder, if it's portable to other DBs.

[boudin]

For MySQL we use github's tool gh-ost https://github.com/github/gh-ost There is a few tech restrictions, but if your database meets that it works great.

I migrated huge tables with absolutely no impact on production

[fabianlindfors]

Your situation sounds just like what I have in mind for Reshape to fix! The backfills performed by Reshape are automatically done in batches to avoid locking.

As you said, Reshape can't call code outside the database as it relies on transactional guarantees for the migrations. Postgres has quite powerful support for procedural languages though so even though Reshape only supports SQL at the moment, there are opportunities to extend that in the future.

Reshape is currently Postgres only and uses a bunch of its features: schemas, updatable views, triggers, procedural functions, transactions and more. I'm sure equivalent things exist in MySQL for Reshape to work with that as well but I'm focusing on Postgres at the moment.