[iJohnDoe]

This sounds very appealing.

Our IDS solution outputs zeek/suricata info to s3 as dns.1234.log.gz, http.1234.log.gz, etc.

Can these files be handled automatically?

[shaeqahmed]

Yes, they would be handled automatically. Data ingestion is supported through S3 or Kafka, where files are picked up and ETL'D into structured Iceberg tables conforming to an ECS-like schema.

Feel free to join our Discord, happy to walk you through the steps and learn about your use case.