[akbo]

Dependency management can be pretty overwhelming for a lot of people entering Python. This is especially true in the data science realm, where many don't have a SWE background. Even after you have selected a tool, it can be easy to use it in a poor way. I have recently written a short article on how I use conda in a disciplined way to manage dependencies safely: https://haveagreatdata.com/posts/data-science-python-depende...

[edsac_xyzw]

Python dependency management of packages using C or C++ behind the scenes is really problematic and sometimes, the installation may fail. In this case, a solution is to use Conda or mini conda which provide many pre-compiled packages and also Clang C++ compiler.

An alternative way to allow people without software engineering background to play with Python data science and machine learning tool may be providing pre built Docker images with everything pre-installed which may save one from configuration trouble.

Docker is also useful for learning about new programming languages without installing anything. With just one command $ docker "run --rm -it julia-image", one can get a Docker image containing a GOLang compiler; a Julia language installation; a Rust development environment and everything else. Docker is really a wonderful tool.

[mariokostelac]

Docker is definitely an interesting tool for that, but my biggest problems is that I have to teach them Docker, which is a totally new layer of abstraction they haven't seen before.

How do you approach this? How technical are people you prepare Docker images for?

[edsac_xyzw]

You don't need to teach docker. All you need is providing a docker image with everything pre-installed such as Julia, R language, Python, numpy, pandas, Tensorflow and maybe Vscode. And also any Linux distribution, then one can just type "$ docker --rm -it -v $PWD:/cwd -w /cwd my-image ipython" For better convenience, it is better creating a command line wrapper or shell script that saves one from typing that such as $ ./run-my-image ipython. I don't prepare anyone, but I guess that if I knew anything about docker and was given a docker image with everything ready and pre-configured and also a shell script or command line encapsulating all docker command line switches, I would find it more convenient than installing everything myself or fighting some dependency conflict or dependency hell. So, docker can be used as a portable environment development. VScode, aka visual studio code, also supports remote development within docker containers with extensions installed per container. I am a mechanical engineer by training, but I found docker pretty convenient for getting Julia, Octave, R language, Python, Jupyter Notebook server without installing anything or fighting with package manager of my Linux distribution when attempting to install a different version of R, Julia or Python. This approach makes easier for getting bleeding edge development tools without breaking anything that is already installed. I even created a command line wrapper tool for using docker in this way that simplifies all those case: $ mytool bash jupyter-image; $ mytool daemon jupyter-notebook ...