1. I still wouldn't put it past developers to lie occasionally.
2. You don't control the test environment. What if the tests pass on the developers machine but not on the CI machines?
3. Difficult to test on multiple platforms. What if the developer uses Mac but you need to support Windows too?
4. Testing can take a long time. Who wants to wait an hour to submit a minor PR? Plus it ties up resources on your machine. CI can automatically scale.
5. There's no way to avoid races between testing and integration (i.e. merge queues).
CI isn't going anywhere. I think the biggest scope for improvement to CI is
1. Only testing things that have changed. Most people don't do that because it requires a build system that properly isolates everything. Basically only Bazel and it's derivatives do this. You can't do it with CMake or NPM or Cargo or ...
2. Make it easier to run CI on your own machine. I don't think there's really any technical barrier to this, it's just people don't usually bother.
This reminds me of a brief discussion/feature-request on the nektos/act GitHub repository (a project that allows developers to run GitHub Actions workflows locally):
> Trust in the developer is required ... but it is likely the case that developers are incentivized to illustrate that they are reliable, careful and trustworthy -- and that should strongly encourage accurate test result signing
100% guaranteed you're going to see "oh I just made a stupid typo that broke one test but I can see what is wrong so I'll just fix it and use the previous test results".
Also since it's entirely based on trust anyway I'm not sure what additional benefit signing gets you. I don't think there's really a way to prove you ran the tests.
Either way, all my other points still stand. CI isn't going anywhere.
> Also since it's entirely based on trust anyway I'm not sure what additional benefit signing gets you. I don't think there's really a way to prove you ran the tests.
It's a reputation and trust-building exercise, essentially - and that's one of the reasons that continuous integration is particularly useful. "Entities X, Y, Z all say that commit <ID> looks good".
> Either way, all my other points still stand. CI isn't going anywhere.
1. I still wouldn't put it past developers to lie occasionally.
2. You don't control the test environment. What if the tests pass on the developers machine but not on the CI machines?
3. Difficult to test on multiple platforms. What if the developer uses Mac but you need to support Windows too?
4. Testing can take a long time. Who wants to wait an hour to submit a minor PR? Plus it ties up resources on your machine. CI can automatically scale.
5. There's no way to avoid races between testing and integration (i.e. merge queues).
CI isn't going anywhere. I think the biggest scope for improvement to CI is
1. Only testing things that have changed. Most people don't do that because it requires a build system that properly isolates everything. Basically only Bazel and it's derivatives do this. You can't do it with CMake or NPM or Cargo or ...
2. Make it easier to run CI on your own machine. I don't think there's really any technical barrier to this, it's just people don't usually bother.