[Annatar]

You assume I am the library maintainer, but I am not. I am just the library user. ergo, no amount of apt-get will fix it for me.

Then take the Apple Computer, vertically integrated stack approach: package all the libraries you require into OS packages, so that the entire installation is encapsulated.

No amount of apt-get will do that if the library/distribution/upstream developer didn't.

How about working with the upstream developers on it? But I agree on this point with you: you, as the user, have a right to expect to be treated with dignity and respect: if the developers aren't providing OS packages for you, and they are on the same platform as you, and refuse to do so in favor of making their lives easier, then that's just wrong.

Furthermore, Docker makes everything work magically for system administrators, and yet you are against it. Why?

Because it doesn't. Docker makes a system administrator's job a living hell: when production kicks the bucket, and the system administrator is called in the middle of the night to troubleshoot the application, he cannot ask one subsystem designed for this purpose to tell it about the software: the software management subsystem. System administrators hate being burdened with artificial dependencies which make developers' lives easier, but make system administrator's jobs living hell. Change and configuration management with Docker is hell, because it doesn't exist. Docker isn't designed to support the change management process, because the developers of it have never heard of such a thing. How would you model the change management process with Docker? What about software lifecycle management, roll a several gigabyte image every time you need to fix your database component? What if you have thousands of such applications that you're responsible for? And why should you be forced to use Docker, when there are zones which provide far superior containerization?

As if that weren't bad enough, if developers delivered their software as modular units -- as OS packages, Docker would be completely and utterly unnecessary. In fact, it's unnecessary now, and only artificial insistence from developers makes it necessary. This is not a good situation.

Dumping a bunch of files into a format foreign to the operating system is never a good solution. OS packaging is designed from the ground up and optimized for managing software. Provisioning technologies like JumpStart, Kickstart, AutoYaST, Ignite-UX and NIM make it possible to go from bare metal to fully functioning systems, ready to accept and serve data. When one has OS packages, all one needs to do is plug them into systems already optimized for that. No need for arbitrary 3rd party solutions.

Packaging was and is, actually, in the strictest sense, designed for software developers, and yet, I have worked with many, many software developers who actively fight even looking at how to make an OS package, for no better reason than learning something new, and yet they'll spring for every shiny new programming language which shows up.

I've single handedly administered north of 200 servers of various makes (linux, windows, solaris), I understand the need for repeatability, and I get it from conda. I can't get it from the underlying package managers. My experience is so different from what you are proposing that I suspect we are coming from different worlds.

I don't know what to tell you. I have almost 73,000 systems across various countries around the globe, and growing daily with no end in sight. I specialized, and have formal education in UNIX system administration (in addition to formal education and professional experience in programming and algorithms). Several decades of professional experience in both, plus database engineering experience, plus large scale system engineering and architecture experience, as you can probably infer from the number of systems. But really, I don't think our worlds are that different, I don't think they're different at all: if it works on almost 73,000 systems, it will work on "north of 200".

[beagle3]

Let me clarify. I managed, on my own, on metal, north of 200 in 2006, as a developer, before all modern orchestration software and even EC2 wasn't even announced until later that year. And when that started taking a nontrivial amount of time, I hired a sysadmin (which is not my job). I didn't bring it up as a pissing contest, just to indicate that I have relevant experience and appreciation of consistency. It's great you have 73,000 in your fleet, but I'm not at all sure you have more work to do than i did with 200 a decade ago.

That said, we do live in completely different worlds. I deliver software that works, repeatedly, on many flavors and historical versions of Linux, some out of vendor support or that never had vendor support, and windows. And does that without inconveniencing the user, and without costing me much in development time as to make it uneconomical.

None of your suggested solutions make any sense in my world. I need to run python 3.5 and a variety of its packages, but the client runs ubuntu 10.4; you suggest I either get the python maintainers to compile it for 10.4, or the ubuntu devs to, which couldn't have happened even when 10.4 was still supported; and then do it for every package I need. Alternatively, you want me to compile and maintain it myself as an 10.4 package, that shouldn't interfere with system python - but also 12.04, 14.04, 16.04 windows and a couple of centos. Which would also make me liable for tracking upstream security for every package I use, rebuild and distribute.

Alternatively, I can ask my customer to install miniconda (one download, one command to execute, regardless of platform), and then install my package through conda (two commands - create env and install). No interference with system packages, no breakage when they upgrade their system to a newer distribution.

Where as in your world you can run a mini Apple to vertically integrate all your packages, take responsibility for security updates for code you don't own, possibly force your customers to a uniform setup (and let go those who don't/can't).

Totally different worlds.

[Annatar]

but I'm not at all sure you have more work to do than i did with 200 a decade ago.

I don't, and that is exactly my point: it's not more or as much work as you perceive it to be, making one's own packages. An experienced packager can get 98% of the software packaged in under five minutes. I do that all the time.

I would have never picked Python precisely because of the pathologies you describe (and which became obvious to me when building and packaging Python). In my experience, your choice of the programming environment led down the rabbit hole you describe.

[beagle3]

What is your language of choice then?

How would you support the breadth of environments in that language of choice? Also, you fail to address the question of multiple incompatible environments and of windows support. Do give examples of how you package everything you produce as deb/apt, rpm/yum, exe/msi/chocholatey in a way that works on multiple OS versions and distributions. Please, concrete examples, not general "I would build it the way Apple would integrate verically". I have explained how I solve my problems, and as far as I can tell, (mostly because you avoid any specific answer to any of my questions) your solution is to specify the clients environment to a t, which is impossible in my world.

Do tell how you repackage e.g. the just-released Python 3.6 with Pillow and 20 other libraries for all these environments in a way that does not in any way conflict with or is influenced by the system Python. Or the same with Java8, or Ruby. or Tcl. Or Perl, or PostGRES, or MySQL, or nginx, or apache, for that matter. And in under 5 minutes. I wish to learn.

I am amused at your choice of words and perception. Life is easy for me, extremely easy, and it is so for my customers who run and administer their own machines. And yet you seem to believe I am tormented, or they are. I can easily support 10 very different environments without thinking, and enjoy the best tool for this job (which happens to be Python), the latest version thereof if I need it.

I perceive your reasoning to be dogmatic, and mine pragmatic. To each his own, I guess.

[Annatar]

What is your language of choice then?

Depends on the problem at hand; these days, mostly control logic in Bourne (not even bash!) shell and an AWK processing / compute core.

Do give examples of how you package everything you produce as deb/apt, rpm/yum, exe/msi/chocholatey in a way that works on multiple OS versions and distributions.

A piece of software I develop always starts out as a Solaris System V package. Always. However, the code is written such that it's completely portable to all other UNIX like systems, which makes it trivial to build an RPM. I then take alien to build DPKG's for debian, and building HP-UX's PSF's is pretty straightforward too.

I do not, and will never support Windows: first, I don't want to support something that I want to end, and second, I don't do consumer space. It's not worth it. Not having to support a braindead platform (no /usr/bin and applications in it without Windows Services for UNIX or Cygwin) like Windows makes the entire thing manageable.

I don't touch Python 3, because it's not backward compatible, therefore I boycott it, and the Python software I use requires Python 2 anyway. I don't do Java, in fact, if I catch anyone using it, that's immediate grounds for dismissal, no questions asked, not that people I work with would ever do such a thing.

PostgreSQL, Apache, and so on: we developed a generic build engine, which builds this software, so that apart from patching / porting it will compile and install the built software under a builds/postgresql-#.#-SunOS-i386[.64], for example. Then I wrote a whole bunch of tools which automate package metadata files' generation, based on command line options, like so:

  % mkpkginfo -d description -n name -v ...

these then become interfaces, which a continuous integration system (or even cron, if you wanted to!) can call. Once these are finalized, they're checked into the source code repository.

There is much more to it than this that I'm not describing, like change requests, configuration management, and so on, for example, configuration packages which, when installed, automatically configure Apache, or any other application, without any human interaction whatsoever.

You seem to me to be in consumer space, writing "fat client" software for the desktop in Python, whereas I'm in the "backend" server space, developing for the infrastructure and web applications.

Life is easy for me, extremely easy, and it is so for my customers who run and administer their own machines.

Perhaps that is because they only have a handful of systems, and perhaps it is because they accept a salad of solutions, like using your proprietary packaging. I (and luckily the employers I used to work for in the past) allow no such salad. The environment is engineered to a T, and that is how we guarantee that what we deliver will JustWork(SM). Zoos and salads are strictly forbidden. It's as if you bought a laundry machine, or a Blueray player: it works, and noone thinks to mess with it, because it's an appliance. Messing with an appliance would be pointless, especially in my problem domain.

I perceive your reasoning to be dogmatic, and mine pragmatic.

And rightly so; my dogmas were shaped by years of extreme suffering in programming "zoos" and being paid to make such "salads" seamlessly work together. I'm the guy who got stuck on the brain detail of figuring out how to convert your proprietary package format into an OS package, integrate that OS package 100% with the operating system substrate, plug it into a provisioning technology like Kickstart, JumpStart(TM) or AutoYaST, and generally make it JustWork(SM). It is a gargantuan amount of work, and all because developers think exactly like you, when in fact, this was the developers' responsibility in the first place. I resent it, and view such software as incomplete. If the developers delivered their software as native OS packages conformant to System V / LSB-FHS and linked with $ORIGIN linker keyword, and if their packages contained code to fully integrate with the OS and start and stop automatically, things would be grand!

[beagle3]

Thanks for the detail, it is interesting --

but it just drives the point that we live in completely different worlds - I cannot turn down Windows using customers (or I could, but I'd have to start a new business), and you apparently can. I believe the vast majority of developers live closer to my world than yours.