[markoa]

Hi, author here. I do come from a team making a CI service, and it took us a lot of time to realize that it'd be useful to have a time-based hard limit for what's good enough. And for anything over that you're obliged to take action to make it better.

E.g. one of our own builds took longer than 10mins at some point. You just assume that it's a function of size of the code and it's normal to rise over time. So my goal is to share that idea & why I think it matters, and get feedback. :)

[alkonaut]

How is the hard limit enforced? By the CI tool as a failed build?

It would be perfect if one could somehow keep the build quick using tools. A hard limit on a build might not be the best way. If the test is taking Limit minus one millisecond, and I add one new test that takes one millisecond, I break the build. But the culprit was really commit yesterday that added a test that takes 9 minutes. The worst thing one can do in CI is somehow flag the build as "not good enough" and point the blame the wrong way.

A tool that treated test quality/perf like any other asset and fails builds based on time limits etc would be perfect. It's really hard to do that especially on shared CI servers because of fluctuating performance.

[markoa]

Yeah sorry for not being clear there, by hard limit I didn't mean that CI should fail the build. But that the number should be a fixed threshold for the team to optimize for. Thanks for your feedback.

[noxToken]

If it's not a function of the size of the code, then what does build time map to? I understand that dependency management, effective testing, proper build configs, etc. also factor into that function. I also understand that size of the code does not have to be a linear scalar, but it seems that a constant hard limit is very arbitrary.

I'm really not trying to crap on your decisions. I just want to know the rationale behind it. There are projects where everything is written in house with 10K lines of code. There are behemoths with 500k+ LoC and numerous dependencies. How does this factor into the decision?

[markoa]

Oh, I meant that "size of the code does not have to be a linear scalar". Also by hard limit I didn't mean that the build should auto-fail or anything (sry English is my 2nd language).

But if we have a rule of thumb, we can know when it's time to start thinking about optimizations.

Code size is certainly a factor but it doesn't have to imply constant time increase. Speaking from experience in web apps, optimization of setup (whatever needs to happen before first test runs) + more parallelism + thinking more about decoupling is usually the right answer.

The point of these discussions is mostly to help people avoid ending up with a single 500k+ LoC behemoth.

[BuuQu9hu]

Hi! Unfortunately, compilers must take at least linear time to compile code, because they must examine every character. Ditto for pretty much any serializer. Also, you'd think that the number of LoC matters, but the choice of language matters too; look how long it takes to compile terse high-level languages like Haskell. Finally, as you probably already know, many steps of a build are non-parallelizeable, and as others have already pointed out, even a task like linking a binary can take multiple minutes at scale.