[jimmyl02]

on the fault tolerance side - if it's really this fast why not just store the data in a high-availability database / data lake then run batch processing? it doesn't seem like this use case requires real-time indexing and searching (which may be why elastic is slower as it supports it)

[arter4]

Sure, but a HA database + batch processing solution and a HA Elastic cluster setup are more or less similar in terms of complexity, number of nodes, RTO/RPO concerns,...