|
|
|
|
|
|
by btown
615 days ago
|
|
|
Oh, that's interesting - sounds like you're implying that the act of making a change in how a database's data is transformed for different parties/outputs, would go through through a review/governance/rollout process, in a similar way (but with even more stakeholders) that a schema migration or code deployment would? And you'd provide GUI and governance controls on top of that? That makes a ton of sense! And I wonder if it would be worth talking to folks who do SOC 2 auditing; if you could say "we can provide a framework that allows you to continue to be SOC 2 certified while letting your developers access real-world data" that would be tremendously valuable. |
|
|
> provide GUI and governance controls on top of that?
Exactly. Data doesn’t exist in isolation. Databases are dependencies of services, and schemas evolve throughout the software lifecycle, often managed by different data migration tools. In large organisations, regular developers usually don’t have direct access to the data sources, and masking rules along with real data sources are often restricted. Schema changes must be validated by the responsible data governance teams to ensure compliance and accuracy.
That’s why we implemented the validate command even in this standalone tool, which checks for schema differences and prevents running a dump if any schema changes are detected with detailed warnings. https://docs.greenmask.io/latest/commands/validate
I once presented Greenmask at an event organized by Percona in Cyprus, and one of the questions raised was: “What if we have a staging database, but instead of cleaning up the database and data, we want to add something to the existing dataset?” At the time, I didn’t have an immediate answer. However, this question inspired me to think, and eventually, I found a solution that at least partially covers this case:
You can restore data in topological order by preserving references and ensuring proper dependency handling (https://docs.greenmask.io/latest/commands/restore/#restorati...)
You can exclude non-critical errors to streamline the process without disrupting key operations (https://docs.greenmask.io/latest/configuration/#restoration-...)
I want to emphasize that this type of software must be flexible and adaptable to meet the ever-evolving needs of businesses… Otherwise, the project is as good as dead.
——————————
> And I wonder if it would be worth talking to folks who do SOC 2 auditing
I’ve had discussions with professionals from Information Security, including those working in SOCs, and you're absolutely pointing in the right direction. At the moment, I’m actively exploring solutions and building a concept. I believe that by 2025, we’ll be able to showcase something new.