Hacker News new | ask | show | jobs
by alpha_squared 2646 days ago
Not sure if I'm missing it in the documentation, but how does it keep in sync with changes in the account?

Edit: OP changed response to remove certain references, presumably because something sensitive was revealed. Changing my response out of respect.

1 comments

Hi, I'm one of the devs on the project.

Good question. As is, this does not keep anything in sync.

To keep the graph in sync with changes in the account, simply set up a cronjob to run `cartography` whenever you would need a refresh. Each sync run should guarantee that you have the most up-to-date data.

Here's how a sync works: when the sync starts, set a variable called `update_tag` to the current time. Then, pull all the data from your AWS account(s) and create Neo4j nodes and their relationships, making sure to set their `lastupdated` fields to `update_tag`.

Finally, delete the left over nodes and relationships (i.e. those that do not have up-to-date `lastupdated` fields). This way the data stays fresh, and you can see this in the [cleanup jobs](https://github.com/lyft/cartography/tree/master/cartography/...).

Makes sense, thanks for the additional details!

Our approach requires us to stay as real-time as possible, so we're actually using CloudWatch events to keep in sync -- the deletes become a little hard after that.

I look forward to the progress of Cartography, though!