[aruntdharan]

You mean the warehouse/app credentials, when you say sync configurations? If so, yes, that seems like a great idea. Infact I think your warehouse credentials are already there in your dbt repo in a specific format. Castled can directly read those credentials from there.

[amcaskill]

That is not what I meant, but also pretty interesting.

I actually mean the 'definition' of the syncs themselves.

I am picturing JSON or YAML that describes the source fields, their mapping to the destination fields, and any other meta about the sync: frequency, number of retries, whatever else that you could configure in the UI

So when I go and update my dbt model to modify one of the tables that I am syncing from, I can make the corresponding changes to my Castled settings file, and release it all as one atomic update to my data infrastructure.

It might be a small number of people who would want something like that, but it's definitely something I would have been excited about when I was running a data team.

[aruntdharan]

We can definitely consider that. But I feel its a lot of config and can be error prone. For instance, source-destination field mapping configs might be complex and have various issues like data type mismatches, typos in field names etc and a user interface is better suited to guide you through the entire process.

But I see value in exporting the config to a github repo after the pipeline is created and thereafter future edits can be done via the github repo. Does that make sense?

[amcaskill]

Yes 100% -- you could also imagine just syncing from the UI to a repo, rather than trying to make the config human-editable. Toggle into a branch in the UI, make edits, and have those committed to the repo by the tool.

Looks awesome, I am rooting for you guys!

[aruntdharan]

Thanks for the input!

[mason55]

> It might be a small number of people who would want something like that, but it's definitely something I would have been excited about when I was running a data team.

Yeah this one certainly depends on the target customer. For me, any tool that didn't have source control integration for configuration would be a non-starter. But it's quite possible that the target audience for this tool doesn't even understand the term "source control".

[bleonard]

Congrats to the Castled team on the launch.

At Grouparoo, this is a primary use case. We have a UI that engineers use locally. This helps gets things right. It outputs a JSON configuration that is checked in. When that is deployed, it does all the syncing.