I don't work at ORY, but I do work on a Zanzibar implementation.
Basically, what you're describing is called "GBAC".
GBAC can be great if you need the full power of a typical Graph API, but Zanzibar-like services are focused on solving the problem of finding a path between two edges[1] and doing everything you can to optimize that operation for latency.
GBAC is flexible, but at the cost of performance compared to something more structured.