[piterrro]

They will compress over the wire, but then decompress and ingest counting billing for uncompressed data. After that, an interesting thing will happen, because they will compress the data along other interesting techniques to minimize the size of the data on their premises. Cant blame them... they're just trying to cut costs but the fact that they are charging so much for something that is so easily compressible is just... not fair.

[jiggawatts]

A part of the problem is that the ingestion is not vector compressed, so they're charging you for the CPU overhead of this data rearrangement.

It would cut costs a lot if the source agents did this (pre)processing locally before sending it down the wire.

[piterrro]

We should distinct between compression in transit and at rest. Compressing a larger corpus should yield better results in comparison to smaller chunks because dictionaries can be reused (zstd for example)