[sporkmonger]

Quite a few years ago, I led a migration off from a legacy logging provider that offered little more than full text search over unstructured text.

Logging at the time was somewhere in the ballpark of 1% of our total common infrastructure spend and widely acknowledged as too expensive relative to the minimal value we got from it with that rudimentary feature set, but it also was nowhere near enough cost to justify doing something about it. We had other observability costs that dwarfed it.

What finally justified the overhaul was that security couldn’t really operate usefully on log data unless we pulled the data out somewhere else like Athena and processed it there. That slowed down security incident response times dramatically.

The migration ultimately benefited the whole engineering organization but it had to be security led to get any traction.