Hacker News new | ask | show | jobs
Show HN: Automated Kafka client and broker optimization (github.com)
2 points by bhatfiel 480 days ago
Sharing this here to see if it helps anyone else or to improve it further. I created this tool to do the following:

Automate initial discovery of configuration optimization of both clients and consumers in a full end-to-end scenario from producers to consumers.

For existing clusters, I run multiple instances of latency.py against different topics with different datasets to test load and configuration settings

For training new users on the importance of client settings, I run their settings through and then let the program optimize and return better throughput results.

I use the CSV generated results to graph/visually represent configuration changes as throughput changes.

1 comments

I'll be online for a few hours and again much later in the day to respond to any questions/comments people have. Thank you for taking a look!