[infinity0]

I hope they improve their software development practises. Packaging SageMath for Debian has been impossible; they use about 100 dependencies and have re-invented their own internal package manager to build all of these with sage-specific patches.

[williamstein]

In 2007, Tim Abbott (before founding ksplice and zulip) beautifully and properly packaged Sage for Debian, and in fact Sage was included in Debian standard! Unfortunately, he stopped working on the project (when he started those projects), and of course we had no money to hire somebody else to maintain what he did. If I had money, it would be a total no brainer to spend it on supporting packaging SageMath for Debian. Your remarks about it being impossible because of our package manager and dependencies aren't correct, because things we the same in 2007.

[infinity0]

OK, more accurately I should have said "much much more cost than is necessary", instead of "impossible".

The point is that, it should not cost significant continual effort to package SageMath for Debian: if SageMath was following good engineering practises, then Tim Abbott's work would still function today, even taking into account necessarily but normal and minimal maintenance costs that Debian volunteers (including myself) would be happy to do for Sage.

[tabbott]

Tim Abbott here :)

The challenge of packaging Sage was primarily around packaging its dozens of dependencies (some of which I had to talk to the authors to fix their licensing terms) and making sure that an up-to-date version of those dependencies was available in Debian. It took about a month of my time to package Sage well for Debian (at the time I maintained over 100 Debian packages for MIT, so I was quite efficient at this).

What killed my Debian packaging effort was that Sage was very large and my package was submitted to the NEW queue (where Debian does copyright review) when all the reviewers were busy with managing a release freeze. So it took more than 6 months for Debian's FTP masters to fully review it, and by the time they did, I had moved from being a grad student to the CTO of Ksplice. It probably would have been just week or two's work for me to update Sage across those 6 months, but running a startup is a lot of work, and I never found the time to do that work :(.

Overall, my opinion is that Sage is well-engineered and not difficult to package given its scale (it has a fantastic test suite, so it was very easy to check if the package worked, and it was easy to get them to merge changes to improve the tooling). The problem is that it's a large project, with a large number of diverse dependencies, new versions of which aren't always backwards-compatible upgrades. If you talk to the folks who package other large projects for Debian, I'd be surprised if you find any that don't involve significant maintenance work.

[infinity0]

More details here: https://wiki.debian.org/DebianScience/Sage

It would be great if you do pay someone to work on this, but please also keep in mind my points about continual costs. To reduce these, Sage upstream (you) does have to change some of its practises.

[iguananaut]

"Sage upstream (you) does have to change some of its practises"

This is also being worked on and there's progress being made.

I'm still pushing to completely separate Sage-the-library from Sage-the-distribution. There's been a little pushback but nothing that can't be overcome with basic configuration management practices.

[iguananaut]

I'm actually one of the few people being paid to work on Sage, and this is one of the tasks I have interest in.

Although my personal work is more focused on Windows support for the time being, this is definitely on the docket. We had a workshop about two months ago in France focused specifically on packaging Sage, and there are some excellent folks from LogiLab who are making serious progress on the Debian packing. I hope to circle back around to that myself after I've made more progress on Windows.

[infinity0]

Great! Could you elaborate what LogiLab are doing? There is creating .debs, and there is creating a .dsc source package for inclusion into the official Debian archives. If you guys want to do the latter, you should talk more to the other people mentioned here:

https://anonscm.debian.org/cgit/debian-science/packages/sage...

The main barrier at the moment, is that sage patches many dependencies. It is better to upstream those patches, not only because it's good engineering practise, but also because it's unlikely that Debian policy (in practise: the admin, infrastructure, and security teams) would allow us to include (e.g.) a duplicate maxima-with-sage-patches in Debian, just to satisfy Sage.

OTOH if you "just want to" create .debs, the task is much easier. But then there's no chance of it entering Debian officially.

[iguananaut]

No, the goal is to have it actually in Debian. One argument that's been made against this is that Debian moves to slowly, and an old (but supported!) version of Sage is not useful to its current core user base of research mathematicians who often need the bleeding edge and/or are developing new code directly in Sage.

My counter argument is that we want to expand Sage's user base beyond a small core of researchers, and improve its usability as a Mathematica replacement for students and some scientists who are less interested in things like bleeding edge combinatorics research (they might be but not necessarily the majority). As a Mathematica replacement Sage isn't there yet, but it's good to get a head start on making easier to package and install, as part of that effort. Like for me, if it can solve some differential equations for me and do some integrals I don't care if the version I got through apt is a couple years old.

As for the upstream issues part of the problem there is that some of the upstream dependencies of Sage refuse to accept patches needed for them to integrate with Sage. That's a long story. I think the best approach there, which has already been tried in past approaches to patching for Debian, is to maintain Sage-specific forks of that software that include the necessary patches (IMO they should also be swappable with the originals via update-alternatives if possible). As far as I know there's nothingf legally preventing that, but more the effort involved in maintaining a fork and a package for that fork.

In the long term, I think, it would make sense to completely replace and rewrite some of the code that these external dependencies are used for. But in many cases there's an enormous amount of work involved, and that would only be possible with significant funding. And quite possibly not worth the effort compared to other ways that effort could be spent.

[infinity0]

Perhaps we should continue this by email - you can contact me at [e-mail]

> Debian moves to slowly [..] My counter argument is that we want to expand Sage's user base [..]

Beyond that - "too slowly" applies only for "Debian stable"; and users that are OK with less stability can use "Debian testing". Usually this is quite bug-free; things only enter testing if it's been in "Debian unstable" bug-free for 5-10 days.

It's also much easier to get your software into Ubuntu, if it's already available in Debian testing/unstable - and that would likely expand your user base quite a lot.

> some of the upstream dependencies of Sage refuse to accept patches [..] [we could] maintain Sage-specific forks of that software [..] swappable with the originals via update-alternatives if possible [..] [or] completely replace and rewrite [it] [..]

Yeah, the situation is complicated. We could try different approaches for each dependency too, and perhaps some of them will change their mind. Debian does (on purpose) make it quite high-cost to maintain forked packages, in the sense that we would have to argue our way through many layers of admins of different systems, to incentivise us to get patches accepted upstream.

When you have time, could you write up the details of the situation on your end? Something similar to the wiki page I posted earlier - or you could also just edit that directly, if you wish.

[goldenkey]

I just tried to get a binary to install it on Windows. Instead they are handing out only OVM files (VirtualBox images.) As much as they might despise closed-source, there are simple things that businesses necessitate - like not being hostile to your audience. I am a fucking programmer, I don't have a problem using VirtualBox. At the same time, it infuriates me that they did not bundle up an MSI or EXE for their windows audience. Now I feel less inclined to even try it, because I already use Mathematica. This basic understanding of "energy to demo something" is the ethos of HN. I am super surprised after reading that PDF to find these simple energy laws of consumerism violated. Aye...I digress...

[williamstein]

I'm sorry. The point of my talk is that Sage is not a viable alternative to Mathematica, etc., for many reasons, e.g., not having a native Windows version. Porting a huge amount of deep technical software from Linux to Windows is the sort of difficult and thankless work that will not get somebody tenure in academia. I tried hard to get funding to get help on a Windows port, and Microsoft donated $30K for this effort back in 2008. However, $30K is not enough to fund such a huge project. In fact, I once met with a bunch of the revolution analytics devs, and learned they were getting millions from Microsoft just to port R to run natively on Windows. This was disturbing, because R is just one of the 100 components of Sage. In my talk, I mentioned the new grant in Europe that is funding the first ever fulltime Sage employee, and it turns out his main job so far is working on a native Windows port of Sage. Unfortunately, though he is incredibly good, he'll probably discover how daunting this project really is. (It's not a one guy for a few months sort of project... And yes, I tried very hard once to port Sage to Windows and failed.)

[iguananaut]

The good news on that front is most of the problem is not Sage itself but some of its dependencies. Of course some of the problem children are core dependencies and can't just be made optional. But fast progress is being made. I have most of Sage running on Windows, currently with an unfortunate Cygwin dependency. However I have hopes to work on native Windows ports of some of the trickier dependencies, notably GAP and Cysignals.

[pmarin]

Sage should run fine in the new Microsoft's Bash on Windows

https://msdn.microsoft.com/commandline/wsl/about

[williamstein]

It doesn't quite work so easily because of some parts of sage use pseudotty's.

[goldenkey]

Thanks for your reply William. I was just frustrated, sorry that I disrespected the work, effort, and time you have put into Sage to get it to where it is today. I will be giving the web version of Sage a try and perhaps contribute to it on an open-source basis eventually.

Cheers.