[jamesblonde]

We built the first open-source feature store for ML, https://github.com/logicalclocks/hopsworks , when every existing proprietary feature store (Uber Michelangelo and Bighead at AirBnb) were shouting about how their DSL for feature engineering was the future.

Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PySpark. The DSL can never evolve at the same rate as libraries in a general-purpose programming language. So, your DSL is great for show-casing a Feature Store, but when you need to compute embeddings or train a GAN or done any type of feature engineering that is not a simple time-window aggregation, you pull out Python (or Scala/Java). I am old enough to have seen many DSLs in different domains (GUIs, aspect-oriented programming, feature engineering) have their day in the sun only to be replaced by general-purpose programming languages due to their unmatched utility.

[bnr]

The article (controversely, maybe) classifies libraries for general-purpose programming languages as internal DSLs. One could argue that Data Scientists working with libraries in Pythin are already using a DSL, just with an escape hatch into the general-purpose world.

I don't think proper vertical DSLs should be made marketed towards people who are comfortable working in a general-purpose language. I see them as a way to help non-technical domain experts work on code instead of specification. Limiting the possibilities of what one can write, like with MPS' projectional editor, is a feature here and not a bug.

[jamesblonde]

A library is not a DSL, despite what the learned authors may claim. In fact, there are a couple of examples of "successful" DSLs in the data world - you could argue that Talend and DBT are visual programming tools for ETL pipelines. Defintely Zapier- which is just for integrating services that have well-defined REST APIs.