[Giroflex]

> Even something as simple as installing a library is a conceptual leap for these people (why wouldn't the software just come with everything needed to work?). > Have you ever tried explaining the various python package and environment management options to someone with a background in Excel/SQL?

I don't understand the difficulty I've often seen voiced against this. Why would a newbie or someone who just wants to get analytical work done need anything beyond installing Python and doing `pip install library`? It's certainly orders of magnitude easier and faster than, say, using a C library. The only trouble I can see a newbie running into is if they want to install a library which doesn't have precompiled wheels and they need some dependencies to build it, but that's rarely an issue for popular packages.

[CJefferson]

Well pip install library needs root, which you probably don't have. So now you have to teach them about making, and acitvating, virtual environments.

Also, they can't easily search through the packages in a nice GUI and just click on the one they want to install.

[Giroflex]

>pip install library needs root

Hmm, not really. It's actually advised against [1].

[1] - https://askubuntu.com/questions/802544/is-sudo-pip-install-s...

[newen]

Pip install needs root on my ubuntu install, my lab's and university's old redhat servers and my windows for linux install. I've had to install anaconda python to get any real work done on all three systems. Anaconda works fine for me but I've not even had to think about anything to install packages in R.

[sdabdoub]

try

    pip install --user

or virtualenvs

[rspeer]

Ubuntu doesn't ship with pip or virtualenv. In fact it ships with a version of Python where the built-in equivalent to virtualenv, pyvenv, is explicitly disabled.

So you have to install extra Python packages, as root. You have to have that Python experience that guides you to install as few of them as you can, just enough so you can get started with a virtualenv, so you don't end up relying on your system Python environment.

And this is really hard to explain to people who aren't deeply familiar with Python. "Never use sudo to install Python packages! Oh, you got errors. We obviously meant use sudo for two particular packages and never again after that."

In the terrible case where you don't have root, you have to ignore Ubuntu's version of Python and compile it yourself from scratch. Hope the right development libraries are installed!

Maybe I'm wrong and there's a method I've overlooked. If there is: please show me how to install a Python package on a fresh installation of Ubuntu 16.04, without ever using sudo, and I will happily spread the good news.

[int_19h]

That sounds like a major problem with Ubuntu, rather than with Python or pip.

On Windows, meanwhile, the standard Python installer gets all this set up properly in like three clicks. Better yet, because it installs per-user by default, "pip install" just works. And if you still choose to install it globally, it will fail, but it will tell you exactly what you need to do to make it work:

    Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: ...
    Consider using the `--user` option or check the permissions.

One can't help but wonder how we ended up in a situation where the most popular Linux distro somehow does Python worse than Windows.

[orbifold]

This is not really thinking though, just a ~3 hour commitment once to figure out how it is supposed to work.

[Sean1708]

Not on a personal computer, no, but the vast majority of managed systems won't let you install anything outside of your home directory. Of course you could install using `pip install --user` but you will inevitably run into problems when something you install locally needs an updated version of something installed on the system.

[mr_toad]

Makes it fun when running on a VM in the cloud which only has a root user. Docker becomes almost essential to preventing errant Python scripts fudging up the system.

[nzjrs]

This is bad advice. Pip should be used in virtual environments, and not to install system packages

[txcwpalpha]

While you're right that it's bad advice, it also highlights the problem with pip that these less experienced people have. The ideal way to deal with Python packages is virtualenvs, but setting up a virtualenv, and then activating it every time you want to use it (or setting up tools to do it for you) is an incredibly huge headache for less experienced people to deal with. R doesn't require that whatsoever.

[jupiter90000]

Neither language requires an isolated dev environment, but it can help with avoiding headaches. As python has things like virtualenv and buildout, fortunately R has 'packrat' available, which provides a similar isolated/reproducible dev environment solution.

https://rstudio.github.io/packrat/

[int_19h]

There's nothing wrong with pip for installing per-user packages outside of virtual environments.

[ekianjo]

And pip s upgrade process for packages is almost non existent, while R supports that very, very easily.

[Giroflex]

Never had any trouble with `pip install --upgrade library`.

Not sure if there's support for upgrading all packages at once, though.

[ekianjo]

There is no such support for multiple packages at once. R can do it though.

[scaryclam]

You can certainly update multiple packages at once using pip. Just use a requirements.txt file, which you should be doing anyway if you're using multiple packages (or just want to be able to reproduce your environment).

[YeGoblynQueenne]

>> Why would a newbie or someone who just wants to get analytical work done need anything beyond installing Python and doing `pip install library`? It's certainly orders of magnitude easier and faster than, say, using a C library.

Except when it isn't. For instance, because some wheel fails to build because you're lacking the VC++ redistributable (or it's not where pip thinks it should be):

  C:\Users\YeGoblynQueenne\Documents\Python> pip install -U spacy
  Collecting spacy
    Downloading spacy-1.2.0.tar.gz (2.5MB)
      100% |################################| 2.5MB 316kB/s
  Collecting numpy>=1.7 (from spacy)
    Downloading numpy-1.11.2-cp27-none-win_amd64.whl (7.4MB)
      100% |################################| 7.4MB 143kB/s
  Collecting murmurhash<0.27,>=0.26 (from spacy)
    Downloading murmurhash-0.26.4-cp27-none-win_amd64.whl
  Collecting cymem<1.32,>=1.30 (from spacy)
    Downloading cymem-1.31.2-cp27-none-win_amd64.whl
  Collecting preshed<0.47.0,>=0.46.0 (from spacy)
    Downloading preshed-0.46.4-cp27-none-win_amd64.whl (55kB)
      100% |################################| 61kB 777kB/s
  Collecting thinc<5.1.0,>=5.0.0 (from spacy)
    Downloading thinc-5.0.8-cp27-none-win_amd64.whl (361kB)
      100% |################################| 368kB 747kB/s
  Collecting plac (from spacy)
    Downloading plac-0.9.6-py2.py3-none-any.whl
  Requirement already up-to-date: six in c:\program files\anaconda2\lib\site-packages (from spacy)
  Requirement already up-to-date: cloudpickle in c:\program files\anaconda2\lib\site-packages (from spacy)
  Collecting pathlib (from spacy)
    Downloading pathlib-1.0.1.tar.gz (49kB)
      100% |################################| 51kB 800kB/s
  Collecting sputnik<0.10.0,>=0.9.2 (from spacy)
    Downloading sputnik-0.9.3-py2.py3-none-any.whl
  Collecting ujson>=1.35 (from spacy)
    Downloading ujson-1.35.tar.gz (192kB)
      100% |################################| 194kB 639kB/s
  Collecting semver (from sputnik<0.10.0,>=0.9.2->spacy)
    Downloading semver-2.7.2.tar.gz
  Building wheels for collected packages: spacy, pathlib, ujson, semver
    Running setup.py bdist_wheel for spacy ... error
    Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\spacy\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'ex
  ec'))" bdist_wheel -d c:\users\yegobl~1\appdata\local\temp\tmpypkonqpip-wheel- --python-tag cp27:
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-2.7
    creating build\lib.win-amd64-2.7\spacy
    copying spacy\about.py -> build\lib.win-amd64-2.7\spacy

    [217 lines truncated for brevity]

    copying spacy\tests\sun.tokens -> build\lib.win-amd64-2.7\spacy\tests
    running build_ext
    building 'spacy.parts_of_speech' extension
    error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
  
    ----------------------------------------
    Failed building wheel for spacy
    Running setup.py clean for spacy
    Running setup.py bdist_wheel for pathlib ... done
    Stored in directory: C:\Users\YeGoblynQueenne\AppData\Local\pip\Cache\wheels\2a\23\a5\d8803db5d631e9f391fe6defe982a238bf5483062eeb34e841
    Running setup.py bdist_wheel for ujson ... error
    Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'ex
  ec'))" bdist_wheel -d c:\users\yegobl~1\appdata\local\temp\tmp8wtgikpip-wheel- --python-tag cp27:
    running bdist_wheel
    running build
    running build_ext
    building 'ujson' extension
    error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
  
    ----------------------------------------
    Failed building wheel for ujson
    Running setup.py clean for ujson
    Running setup.py bdist_wheel for semver ... done
    Stored in directory: C:\Users\YeGoblynQueenne\AppData\Local\pip\Cache\wheels\d6\df\b6\0b318a7402342c6edca8a05ffbe8342fbe05e7d730a64db6e6
  Successfully built pathlib semver
  Failed to build spacy ujson
  Installing collected packages: numpy, murmurhash, cymem, preshed, thinc, plac, pathlib, semver, sputnik, ujson, spacy
    Found existing installation: numpy 1.11.0
      Uninstalling numpy-1.11.0:
        Successfully uninstalled numpy-1.11.0
    Running setup.py install for ujson ... error
      Complete output from command "c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, '
  exec'))" install --record c:\users\yegobl~1\appdata\local\temp\pip-ibtvwu-record\install-record.txt --single-version-externally-managed --compile:
      running install
      running build
      running build_ext
      building 'ujson' extension
      error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
  
      ----------------------------------------
  Command ""c:\program files\anaconda2\python.exe" -u -c "import setuptools, tokenize;__file__='c:\\users\\yegobl~1\\appdata\\local\\temp\\pip-build-7o0roa\\ujson\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --recor
  d c:\users\yegobl~1\appdata\local\temp\pip-ibtvwu-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\yegobl~1\appdata\local\temp\pip-build-7o0roa\ujson\

Now that's newbie scary.

Note that this is just one case where I was trying to install one particular package. I got a couple more examples like this in my installation diary, notably one when I tried to install matplotlib, this time on Windows Subystem for Linux, a.k.a. Ubuntu, and hit a conda bug that meant I had to use an older version of QT until upstream fixed it and other fun times like that.