|
|
|
|
|
|
by bayesian_horse
1201 days ago
|
|
|
You already set up that workflow. That can be a major PITA. Also I don't think this is as much about the "compute" of the background job as much as the nodejs app saying "take care of this for me please, got this? Don't bother me again until I ask for the result". Endpoint handlers are flaky. They aren't guaranteed to run to completion even through no fault of their own. They should ideally have short and predictable run times, otherwise scaling is much more painful. That's why people separate out certain stuff into those background jobs. And getting that just right, scale it and so on can be quite a bit of work, but yes, something like Cloud Functions or Lambda would serve very similar functions. If you set up all the CI/CD pipelines to make that work. |
|
|
I've been "abusing" GitLab CI pipelines a lot for running periodic background jobs that I'd like to have separated from the "regular" backend and without worrying much about deployment.
It supports Cron Syntax for scheduling, manual re-runs, provides secret management and download of artifacts, alerts me on failure and allows me to easily use the docker images in our registry.
Sure, I'm not storing anything in a database or calling the pipeline via an endpoint. But if that's needed, it should probably be "part of the regular backend".