Hacker News new | ask | show | jobs
by rockwotj 702 days ago
> delete crap from topics

You have to rewrite the whole topic to do this right? (or do some hacks with compaction if you have unique keys)

1 comments

Or just advance the offsets for that partition beyond the problematic data, or adjust retention, or both.
There is also the deleteRecords API specifically for this. It's easier than the retention shrink -> increase dance, as it is a single API call and retention does not kick in immediately. The log segment must roll for retention to apply, either due to size or time.

https://kafka.apache.org/11/javadoc/org/apache/kafka/clients...