[hopel]

Do you store raw data ingested from Kafka directly in S3 or have an intermediate database for hot data?

[paladin314159]

The raw data is stored directly into S3 every hour, but there are multiple systems that process the data directly from Kafka in order to produce the real-time pre-aggregated views.