Hacker News new | ask | show | jobs
by frank_nitti 701 days ago
Definitely true. I’ve also seen even the most technical people benefit greatly from the artifacts produced by CI tools.

When each commit (or each RC etc) has automated regression testing for output quality, performance, or XYZ metric critical for your business, the artifacts/reports from those code changes can be stored and audited when unexpected things happen during a release crunch.

We’ve had teams that shrugged off integrating with our CI tools, then a ”final” manual quality check on their release builds’ output showed significant regressions in completeness/accuracy. They scramble all their troops to run through the merged commits to find the culprit(s), and jeopardize delivery.

A basic CI setup could have posted perf/recall/whatever stats to their PR thread for each merged change, or better yet block the merge at some threshold for regressed metrics.

I definitely had the same view initially of CI, similar to unit test suites or verbose structured logging… things I viewed as complete overkill. What I learned was that these things DO still feel like overkill until the moment you really need the ability to reliably audit how each change affects an evolving large system

1 comments

"When each commit (or each RC etc) has automated regression testing"

This has a lot in common with old IBM batch processing essentially designed to consume rented resources.

When the teams struggle to get a baseline suite working, adding k8s is not helpful.