Hacker News new | ask | show | jobs
by michalc 1195 days ago
I've never been sure if it's too much of a hack, but I've used GNU parallel in Docker containers as a quick and easy way of getting multiple processes running for web applications.

And with the `--halt now,done=1` option (that I think is relatively recent?) it means that if any of the parallel processes exit, parallel would exit itself, the whole container will shut down, and external orchestration would start another one if needed.

2 comments

Cool tip thanks for sharing! I love letting process crash *when possible* on failures so the OS restart them for me versus trying to handle it manually at process level.
I've used Supervisor pretty successfully for this as well: http://supervisord.org/

Example of installing it in a Debian/Ubuntu container during container build, here's an example Dockerfile:

  RUN apt-get update \
      && apt-get -yq --no-upgrade install \
          supervisor \
      && apt-get clean \
      && rm -rf /var/lib/apt/lists /var/cache/apt/*
Then it's possible to create a configuration file, for example /etc/supervisord.conf, to specify what should run and how:

  [supervisord]
  nodaemon=true
  
  [program:php-fpm]
  command=/usr/sbin/php-fpm8.0 -c /etc/php/8.0/fpm/php-fpm.conf --nodaemonize
  stdout_logfile=/dev/stdout
  stdout_logfile_maxbytes=0
  stderr_logfile=/dev/stderr
  stderr_logfile_maxbytes=0
  
  [program:nginx]
  command=/usr/sbin/nginx
  stdout_logfile=/dev/stdout
  stdout_logfile_maxbytes=0
  stderr_logfile=/dev/stderr
  stderr_logfile_maxbytes=0
And finally it can be run inside of the container entrypoint, along the lines of this in docker-entrypoint.sh:

  #!/bin/bash
  echo "Software versions..."
  nginx -V && supervisord --version
  
  echo "Running Supervisor..."
  supervisord --configuration=/etc/supervisord.conf
Here's more information about the configuration file format, in case anyone is curious: http://supervisord.org/configuration.html

It should be noted that this package will bring in some dependencies, though, which may or may not be okay, depending on how stringent you are about space usage and what's in your containers, example for a Ubuntu container:

  The following NEW packages will be installed:
    libexpat1 libmpdec3 libpython3-stdlib libpython3.10-minimal libpython3.10-stdlib libreadline8 libsqlite3-0 media-types
    python3 python3-minimal python3-pkg-resources python3.10 python3.10-minimal readline-common supervisor
  0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded.
  Need to get 6905 kB of archives.
  After this operation, 25.7 MB of additional disk space will be used.
(just found the piece of software itself useful for this use case, figured I'd share my experiences)

My problem is that it's not always immediately clear how software that would normally run as a systemd service could be launched in the foreground instead. It usually takes a bit of digging around.

I have previously thought a bit about using something like Supervisor. And if I was running something a bit closer to the metal, with no other infrastructure to restart stuff, then I would be much more pro.

But if inside Docker when something else already has the job of restarting things if they fall over, then it feels a bit over complicated in that there are multiple ways of doing the restarting. Plus, I think there is a touch more visibility - it's all just command line arguments to parallel:

    parallel --will-cite --line-buffer --jobs 2 --halt now,done=1 ::: \
        "some_proc some args" \
        "another_proc some more args"
This is pretty crafty. I do not know supervisor well enough - if one of the services fail, can you engineer supervisor to also crash so that it would bubble up to the container infrastructure? My understanding is that standard supervisor would let the process die and/or restart the service.
Supervisor allows you to have event listeners (e.g. for processes quitting/crashing), so you can use those to achieve that and kill supervisor itself. Here's an example of people doing just that: https://gist.github.com/tomazzaman/63265dfab3a9a61781993212f...
Neato. Do not have an immediate use case for this, but definitely something I will consider for the future.