[nemo44x]

I think you may want to check your mappings/templates. There are a lot of data types for this kind of data and they don't rely on the inverted index that you would use for searching fields. Lucene, which Elasticsearch is built on, has a feature called "doc values" that stores data as column-oriented fields. This makes for the fast aggregations, sorting, and grouping for numeric and text fields.

One of the main strengths of Elasticsearch is that you can use it for searching and aggregating in a single query. But you need to ensure you are searching on fields that are indexed for search and sorting/aggregating on fields that are indexed for that.