DAGs aren't very scalable, because like with git, you have to store the whole history.
In some scenarios, you can rebase/snapshot to clean up history, but these usually require a type of centralization or consensus, which defeats the point of using something like git.
As a result, DAGs can only be used to replace a subset of apps/tools out there.
The most important/used apps, though, are indexed lists. Things like:
- Google rankings
- Reddit homepages
- AirBnB listings
- Ubers nearby
If you're updating a geo-index 100s of times a second, like in the case of cars' GPS locations, then you're just wasting resources with a DAG that you'll never use and wind up bottlenecking the system, preventing scaling beyond a certain threshold. I've dealt with this in practice, and it was no fun.
We switched off DAGs, and our biggest production deployment now has been with 15 million monthly users. This is far far far beyond the scale of any of the previous systems.
In some scenarios, you can rebase/snapshot to clean up history, but these usually require a type of centralization or consensus, which defeats the point of using something like git.
As a result, DAGs can only be used to replace a subset of apps/tools out there.
The most important/used apps, though, are indexed lists. Things like:
- Google rankings
- Reddit homepages
- AirBnB listings
- Ubers nearby
If you're updating a geo-index 100s of times a second, like in the case of cars' GPS locations, then you're just wasting resources with a DAG that you'll never use and wind up bottlenecking the system, preventing scaling beyond a certain threshold. I've dealt with this in practice, and it was no fun.
We switched off DAGs, and our biggest production deployment now has been with 15 million monthly users. This is far far far beyond the scale of any of the previous systems.