[sandGorgon]

genuine question - is nobody using anaconda/conda in production ? I have found the binary install experience in conda far more pleasant than in anything else.

Going forward, the trend is going to be pipenv+manylinux (https://github.com/pypa/manylinux), but conda is super pleasant today

[beagle3]

I use miniconda in production, and it's awesome. It's on par with (or even better than) npm except perhaps on the number of packages in the repository, supports pip, does everything I need and then some.

I'm baffled myself at the anaconda-blindness in the general crowd, which is evident every single time this comes up for discussion.

[virusduck]

What happens when there isn't a conda recipe for some package or inexplicably some dependency? Do I go back to pip? sudo pip ;) ? Use virtualenv?? Nothing is ever solved.......

[kalefranz]

> What happens when there isn't a conda recipe for some package or inexplicably some dependency?

You go contribute it on conda-forge? The conda team is also actively working on improving some of these problems specifically for python users. When you create a new conda environment with python in it, we put pip in it for you too. In a way, we're implicitly encouraging you to use pip along with conda, and yet it's not a seamless experience. So https://github.com/conda/conda/issues/7053 is a current effort. Eventually, we're working toward conda being able to install wheels directly (at least pure-python wheels at a minimum), so that packages available on PyPI that conda-forge or Anaconda haven't built out yet can still be installed with conda.

> Do I go back to pip? sudo pip ;) ?

If you're putting `sudo` in front of `pip`, you're probably doing it wrong ;-)

[superbatfish]

Indeed, using pip from within a conda environment is trivial.

But the preferred solution is to make a conda package for yourself, and it's really quite simple. You can host it from anaconda.org, or from a local directory for crying out loud.

[infinite8s]

Why are you moving away from conda going forward?

[mkobit]

Not OP, but we would like to move away from it as well.

- Breaking behavior between minor versions (https://github.com/conda/conda/issues/7290)

- Environments not actually being isolated (https://github.com/conda/conda/issues/448)

- Can't create environments in long paths (https://github.com/conda/constructor/issues/156)

Those are just a few I can remember. We unfortunately have not found a strong replacement.

[kalefranz]

> Breaking behavior between minor versions

See https://github.com/conda/conda/issues/7248 for where conda intends to head in the future on the environment.yml issue.

> Environments not actually being isolated

That's actually a really sticky issue, and one that's more about the python interpreter itself rather than anything conda is doing. More recent discussion at https://github.com/conda/conda/issues/7173. Yes, we can change the default behavior of the python interpreter. Either way though, we'll be making a group of people mad.

> Can't create environments in long paths

Of course you can. `conda create` works well with long paths on unix systems (Windows is more difficult, but we're working on that too). What you're bumping into in that issue is that the constructor installer builder isn't (right now) compatible with longer paths. The solution really is to get conda bootstrapped onto your system, and then just use that one conda. You don't need full miniconda installations scattered all over the place. One easy way to do it is bootstrap conda into `/opt/conda` and then symlink `/opt/conda/bin/conda` to `/usr/local/bin/conda`. Now `conda create` whatever and wherever you want.

> We unfortunately have not found a strong replacement.

Conda definitely isn't perfect, and it's far from a "done" project. One thing we do have at this point is years of battle-hardening with something like six million active users blanketing all sorts of operating environments. With conda-forge being as strong as it is today, I'm not sure anything else like it really exists. Nix and pkgsrc are probably the closest alternatives.

[sandGorgon]

Quick question - has there been an attempt to make conda and official PEP and make conda-forge part of the Python Foundation (instead of a private company).

I'm trying to figure out why is all this new manylinux PEP stuff "inspired by conda" and is not actually conda.

Is this a situation like grsecurity vs Torvalds? How does the situation change now that BDFL is gone ?

[kalefranz]

> Is this a situation like grsecurity vs Torvalds?

Ha! Conda and Anaconda, Inc. are _not_ like grsecurity. Starting with the fact the Conda is BSD-licensed.

Recounting some of the history will probably provide the context you're looking for.

It's my understanding that the birth of Anaconda (the distribution) and Conda goes back to this statement by Guido regarding package building and installing at a PyData summit in 2012:

> It really sounds like you guys' needs are so unusual compared to the larger python community that you're just better off building your own.

https://www.youtube.com/watch?v=QjXJLVINsSA&t=59m10s

This predated PEP 427 (Wheel Binary Package Format 1.0).

Of course both conda and pip/wheels have evolved immensely since 2012, and their evolution has been guided by different constraints. Elsewhere in this thread I spoke about how pip/wheels/PyPA packaging has as it's primary target the site-packages directory, and then branches out from there when necessary. Conda's primary target is the prefix root. PyPA's mandate is to build, install, and manage python packages, which makes something like 'pip install python3' out of scope. Things like 'pip install postgresql' and 'pip install r' are _for sure_ out of scope. Conda has the luxury of being able to install and manage all packages, not just python packages.

Regarding the creep of wheels toward conda packages, and especially the static embedding within wheels of compiled "system" libraries, I question whether it's actually to the benefit of python _users_. No doubt the wheel "binary" format is a huge improvement over eggs for pure-python packages. But manylinux wheels are often built going beyond just compiled python extensions. Rather than describing these "system" dependencies in metadata, and then having pip ensure these system dependencies are present before installing the package, these wheels now implicitly cross the python-only line that PyPA in other cases holds. There are real consequences for users, with the result being that it pushes failure mechanisms back rather forward.

So, because of the differences in scope, my guess is that the python community would think it inappropriate to make Conda an official PEP. Conda will probably someday install wheels (at least pure-python ones); maybe someday pip can reach out to conda or apt-get or yum to ask for non-python dependencies to be installed.

> and make conda-forge part of the Python Foundation (instead of a private company)

Conda-forge is actually a community-driven organization completely independent of Anaconda, Inc.

[craigds]

Binary wheels are a nightmare in terms of knowing what you have installed. We disabled them in our builds when we discovered a few packages had vendored old, terribly-insecure static versions of libxml2 (and who knows what else) which silently replaced using the system version (which received OS security updates). We only found out when we hit one of the behavioural quirks of that version and spent ages investigating it with gdb.

If you're using manylinux wheels you're probably using static libs you have no idea you're using. That way lies madness.

I havent used conda so maybe this doesn't apply there somehow.

[sandGorgon]

> my guess is that the python community would think it inappropriate to make Conda an official PEP.

will you guys try ? I really, really, really hope you do.

[mkobit]

Thanks for the reply. I wrote that post earlier this morning after some other frustration, so I'm sorry for the negative tone.

Despite some of our issues with it, Conda has worked well for us both as a development team and in production for quite some time.

> six million active users

That's awesome to hear, and one of the reasons we haven't seriously invested in a replacement.

[sandGorgon]

we are NOT moving away. In fact, for all its faults, conda has been the most pleasant experience. We publish our own internal packages as conda packages.

I was commenting about manylinux becoming an official PEP - https://www.python.org/dev/peps/pep-0513/ - could eventually end up supplanting conda.

I wish they had adopted conda itself. Because manylinux was clearly inspired by conda.

"Instead, we define a standard subset of the kernel+core userspace ABI that, in practice, is compatible enough that packages conforming to this standard will work on many linux systems, including essentially all of the desktop and server distributions in common use. We know this because there are companies who have been distributing such widely-portable pre-compiled Python extension modules for Linux -- e.g. Enthought with Canopy [4] and Continuum Analytics with Anaconda [5].

Building on the compatibility lessons learned from these companies, we thus define a baseline manylinux1 platform tag for use by binary Python wheels, and introduce the implementation of preliminary tools to aid in the construction of these manylinux1 wheels."

[rb808]

Agreed its the most practical solution for most people. Its also a shame that it shows just how unreliable python packaging is.