|
Author here- thanks! You are totally correct, if Fivetran fails on any of the tables then we'd likely see issues in the tests. My thinking was this was ok, as I'd prefer my tests to fail than me blindly continue with missing data. Obviously a more sophisticated solution would be to page directly on this, but in the name of minimalism, it seemed like an ok solution. There's also a Fivetran audit table that I considered using, as it contains data like this: ```
{
"id": "9b21ce1e-6889-4e3f-ab8f-889425d1a9dd",
"_fivetran_synced": "2022-02-04 00:11:27.413 UTC",
"done": "2022-02-04 00:10:57.690 UTC",
"message": null,
"progress": "2022-02-04 00:10:55.757 UTC",
"rows_updated_or_inserted": "3",
"schema": "core_production_public",
"start": "2022-02-04 00:10:57.576 UTC",
"status": "Ok",
"table": "incidents",
"update_id": "013c201d-7fca-44cb-a5ee-93ba3e74ae5e",
"update_started": "2022-02-04 00:10:55.756 UTC"
},
``` But figured that would have the same issue. That said, I think this is a valid approach for anyone with the more sophisticated alerting! |