[avabuildsdata]

the Go choice makes a lot of sense for this. i've been wiring up government data sources for a different project and honestly the format inconsistency between agencies is always the real headache, not the actual processing

curious about the 23 tools though -- are those all invoked through one Gemini orchestration pass or is there a routing layer picking which subset to call per detection? feels like that'd stack up fast latency-wise

[mapldx]

Not all 23 get invoked in one pass. The system runs 4 different types of cycles, each with its own Gemini call, and within a cycle the model picks a subset of tools based on the context rather than fanning out to everything.

Over the last week, the median ends up being about 6 tool calls across 4 distinct tools per cycle.

Latency-wise, median completed cycle time is about 37s overall. The heavy path is FIRMS: about 135s median / 265s p90 over the same window.

It runs asynchronously in the background, so the web UI isn’t blocked on a cycle finishing, though cycle latency still affects how quickly new detections get enriched.