Super promising. What data types can Veilstream handle? Like can I mask nested jsonb, uuids, IP addresses, arrays? Would be wild if adding new filters was fast enough to support weird internal schemas or bespoke pii.
- jsonb : kinda, we do static json replacement, with more complex rules on the horizon, where you could replace some regex-like path with a random func.
- uuids: no, but I should. adding to my list :)
- ip addresses: yes ip4 and ip6, but I want to go further and let you configure the replacement ips to be within specified cidr blocks
- arrays: again, not yet. Do you mind if I ask the use case? Arrays are commonly done as single rows and foreign keys/look ups, which we can do.
We've internally got the path for adding new filter types (dashboard configuration, api layer storage, and proxy rule implementation) pretty optimized. it takes us a day or two to add simple requested filters. longer for more complex ones.
Beauty. For arrays was thinking of semi-structured stuff in audit logs or embedded tags, but yeah, can probably reshape upstream. Awesome to see the path to custom filters is already so streamlined.
we were considering allowing the user to inject stored procedures themselves, and then use those, but currently, we're opting to implemented them ourselves, so we have better control over the user experience. In the future, for very custom stored procedures, I think we may allow the custom path.
- uuids: no, but I should. adding to my list :)
- ip addresses: yes ip4 and ip6, but I want to go further and let you configure the replacement ips to be within specified cidr blocks
- arrays: again, not yet. Do you mind if I ask the use case? Arrays are commonly done as single rows and foreign keys/look ups, which we can do.
We've internally got the path for adding new filter types (dashboard configuration, api layer storage, and proxy rule implementation) pretty optimized. it takes us a day or two to add simple requested filters. longer for more complex ones.