I wish <my favorite language> also had this. However, containers work. Not sure why they do java though, i thought java has been happy with jars so far.
If anybody is thinking about Java, check Graal out. I was able to successfully create fast, completely static ELF binaries of real JVM applications from fat JARs (not in the fake self-executable archive way).
It's still early days in terms of what's available in the runtime (e.g. the AWT subsystem is missing), but it's pretty damn impressive anyway.
We're struggling using containers for our Python and Javascript applications. I think we're using venvs and node_modules and we oughtn't (or we shouldn't keep them in our project directory where they sometimes--but not always--get overwritten by source code volume mounts).
It can be a little tricky to do things right here.
There are a few things of note:
1. The .dockerignore file can be used to prevent use of your `node_modules` folder during `docker build`, even if you have something like `COPY . .` in your Dockerfile.. This can let you create a new `node_modules` folder from your lockfile as part of the docker build process to create an image for testing/deployment.
2. You can maintain separate Dockerfiles and such for development and for release, so e.g. for development you might use volumes and not copy things in, but for release you wouldn't use volumes for source code.
I can't tell from what you said, but it seems like one of those two tips might be relevant.
It's okay to have a development setup which doesn't use containers and then use containers for deployment (so long as you have an integ or staging environment that uses containers) as well. It's quite reasonable to have a venv during development, but inside the docker image to not use a venv at all since things will already be reasonably isolated inside the container's fs.
Thanks for the suggestions! I had to check, but we do both of those things. To make sure that node_modules aren't overwritten, we mount an empty named volume to the node_modules directory. I guess mounting the named volume directory somehow causes the node_modules directory from the base image to appear inside of the bind-mounted directory. We do the same thing for venvs as well.
We use venvs inside of the Docker container because we use pipenv and apparently pipenv's support for installing to the system is buggy and/or idiosyncratic.
Named volumes are hacky to no end and you shouldn't be using them to make sure it's empty; you should be using `.dockerignore` or being careful about what you copy in.
The pipenv bit is probably more reasonable, and I'm afraid I haven't used it enough to be sure what rough edges are likely there.
Sorry if I'm a little dense; that doesn't seem like it solves any problems. It just says "use virtualenv and/or docker". I guess I was hoping for "how to manage dependencies in a localdev-friendly way for a Python/Docker app" or something.
We just use a requirements.txt [1] for each service and run pip with the -r flag in the dockerfile. No Virtualenv in the container.
Most of the time we run these containers with docker-compose locally, but sometimes we want to run the service outside the container. For that we create a virtualenv outside the folder where we keep the source for the service. The reason for that is that we don't want to accidental copy the virtualenv in to the container. It wouldn't do much there but it would increase the container size.
You can do this easily with just "python3 -m venv", but there are some tools that help with that as well. I personally use pyenv-virtualenv [2] which just keeps all virtualenvs in ~/.pyenv/versions/<env>, but there is also conda [3] and virtualenvwrapper [4] which also store the virtualenvs in a central directory.
I'm not sure if I understand correctly. Are you saying that you run venv commands as part of the build step for your containers (i.e. a RUN line in you Dockerfile) and then pip installing modules in another build step?
No, we use pipenv instead of pip, and pipenv manages a venv. There are `--system` flags, but I guess support for installing things to the system is a little buggy or idiosyncratic or something so we use the venv behavior. I'm not sure why we tell pipenv to install the venv in the project directory though. We do the pipenv install as a RUN line in our Dockerfile. Does this answer your question?
Yeah, I think I have an understanding now. We use tagged Docker base images with the app dependencies baked in so you can just use a FROM your Dockerfile and know you have the dependencies in the image and you just add and run your app code.
Plop a requirements.txt file down in each repo. Use direnv or a similar tool to automatically manage virtualenvs on the local dev machines. In your container, don't use virtualenv; just add requirements.txt to the container and run pip install -r requirements.txt
Have you tried pipenv? It creates a lock file that hashes all your dependencies so it makes sure you get the right one when you deploy.. And it automatically creates the virtualenv if its nog there when you install them.
We do use pipenv. I like the idea of it, but I think we ran into issues with where the pipenv lives. We used to use `--system`, but that seemed buggy and/or idiosyncratic, so we started using a venv and then we started using PIPENV_VENV_IN_PROJECT=true (for some reason) and now it's causing other issues. It seems unclear how it's supposed to be used with Docker containers, but maybe we're just overcomplicating things?
It's still early days in terms of what's available in the runtime (e.g. the AWT subsystem is missing), but it's pretty damn impressive anyway.