[daddykotex]

I agree with you, and I've got a question as well.

I'm wondering how long does it take to execute the ES refresh on a search query when the Shard was marked as dirty?

If the search requests are mostly real time, I suspect this is really short, but if the Shard ingest new messages for a while (let's say 50 minutes) and it's marked as dirty, a search query would ask ES to refresh 50 minutes worth of documents before running the actual query.

As it shown to be a problem? Is the refreshing time growing along with the number of documents inserted since the last refresh?

[jhgg]

Good question. So far we've noticed the refresh time to be negligible (worst case in the tens of milliseconds). It's worth noting that most of the cost of doing a search on Discord is in pulling the message context from Cassandra to provide enough data to render the results in the client.

[daddykotex]

I'm impressed, thanks for answering.