Hacker News new | ask | show | jobs
by ragulpr 516 days ago
This pattern is really meaningful conceptually but the tricky thing is to not create a mess in the process.

If it's too easy to branch people will do so and the (knowledge) economics scale disappears (and we'll have a mess).

If the common data definition is too hard to branch from no experiments will happen (slow).

I think most tech for this seem to make it too easy, and in the process injecting a bunch of dependencies that makes it slow and harder to access. May have changed since I last looked.

I found that the simple pattern of versioned paths/table names as `s3://mybucket/mystage/version=42/` or `my_table_v42` puts a high enough evolutionary cost on branching (as consumers need to explicitly adapt) while it also doesn't have the costs associated with using special tech (legacy/lock in/dependencies).

It's also searchable on github/slack/etc if done right.