|
|
|
|
|
by codingmoh
424 days ago
|
|
Dagster is definitely one of the more polished orchestrators out there, and I like how it embraces the idea of resources and custom DSL-style components. It’s also cool that you’re using it at Magenta/Telekom at scale. The only caution I have is that many orchestrators (Dagster, Airflow,...) are typically task-level or graph-level frameworks: they do a great job of linking up big stages of a pipeline, but I’ve found that you still end up writing quite a bit of manual or ad-hoc code. That’s not necessarily a knock on Dagster—it’s just the reality that orchestrators focus on coordinating tasks, rather than giving you a row-by-row DSL with robust side-effect semantics. Still, those links you shared look super useful for seeing how Dagster can be extended with domain-specific abstractions. Appreciate you pointing those out — I’ll check them out |
|