Another invisible problem with embedded-DSLs (like maybe building within Yaml) is the inability to break the host language’s constraint, limiting utility of the desired DSL
But the only functional constraint is just how it looks when it is serialized. You can write any language’s AST to yaml; it will look like yaml, but the representation at least allows for Turing complete behavior.
I do have issues with how yaml deals with strings, but it’s the best popular format I think I can use, and seems appropriate for developers.
You can write any language's AST to YAML, but not the other way around.
YAML is extremely limited and a very bad choice for DSLs. CircleCI got it wrong. Their configuration files are extremely verbose and hard to abstract over - you have to use orbs and those come with a lot a of flaws. I'm a user for many years now and it's diminishing my productivity.
The solution isn't so hard - even Jenkins got it right. Use a regular programming language (maybe the one of your choice or one that is well suited for DSLs like Scala or Clojure) and then just expect your developers to build a datastructure in that language. This datastructure is your DSL.
Your users get a lot of stuff for free, including fully fledged code-completion and compile-time safety (if you use a statically typed language). It will also be easy to abstract over commong logic and even share it if the language has any kind of module/library system which almost every language has.
Definitely this is a very fair criticism to my comment. I don’t fully agree yaml is bad, - I think it depends on use case But I appreciate you bringing up the other side; I agree somewhat that CircleCI is not as easy to use as it could be due to their choice.
FWIW, the most recent thing I built that could be considered a DSL is how we define roles where I work. It’s a lot like IAM policies. I wouldn’t want to run arbitrary code when parsing these documents, and I think yaml suits it well.
> I wouldn’t want to run arbitrary code when parsing these documents, and I think yaml suits it well.
I understand your point that it might be harder for you to run arbitrary code. Depending on where it runs you need to make sure to isolate it.
But other than that, I think your imagination isn't correct. Because even if I want to define something like permission-policies, there is usually a lot of repitition that I'd like to factor out. And doing that with YAML is most often impossible.
But if you still decide to go with a non-programming language for configuration, then have a look at dhall: https://dhall-lang.org/
It allows a lot of logic/abstraction through functions and imports, is typesafe and also has builtin YAML generation. Even the few code examples at the page that I linked above show how you can factor things out easily there.
I do have issues with how yaml deals with strings, but it’s the best popular format I think I can use, and seems appropriate for developers.