[Beltalowda]

This is already entirely doable: just create some executable "test" program in $language_of_choice (shell script, Python, compiled C binary if you really want to) and run that in the CI. You're still going to have to need a wee bit of CI configuration (usually YAML) to tell it which containers to run and whatnot, but this usually isn't all that much.

[ithkuil]

Yep. CI systems offer some extra features. If you don't use them there isn't really a lock in.

From the top of my head:

1. Parallelization. 2. Capture of build artifacts, which can also be useful for logs of non-linear complex tests such as dependencies of e2e tests. 3. Secrets for release or artifact uploads to third party repos (e.g. docker repos) 4. Caching.

There may be more. Once you sprinkle these things left and right in your CI config, it becomes hard to move to another, even if the bulk of the actual tests you run are just "make test"

[Beltalowda]

You don't really need all that much CI-specific stuff for most of that, except maybe the secrets, although you will end up duplicating some CI features if you choose not to use them, but that's usually not too hard.

[ithkuil]

Long time ago I implemented my own "CI" system. The basic idea was that by putting a make wrapper in the MAKE env variable I would intercept recursive makefile executions and I would spawn tasks in a message queue. Workers would pick up the messages and perform a fast checkout from a hot git repo cache in each node (using git --references). It worked as a charm. The exact same makefiles would work also locally out of the box