[D-Machine]

Sorry if I was unclear, but GP is correct, you are misreading quite deeply.

I am saying that because it is much harder to find good documentation on using MATLAB on HPCs, a lot of computations on HPCs that use MATLAB are highly wasteful compared to if they had been written using a language and/or tools that make it much easier to use HPC resources more efficiently. I was NOT in any way saying that "universities are wasteful".

[esalman]

Fair. My point though is that in scientific research, accuracy of the results come first. It does not mean anything that research was performed efficiently if the results are incorrect.

[D-Machine]

I dunno, since research has to be funded, and typically HPC clusters also have to allocate compute resources, the clunkiness of things like MATLAB actually does translate to wasted and inefficient funding.

All other things being equal, suppose two research programs are proposing to study roughly the same thing (say, some novel optimization or basic stuff on something like simple neural networks; and let's pretend this is some years ago when people still actually reached for MATLAB neural net tools). If both request significant compute allocations, and I see one is planning on MATLAB, and the other on PyTorch Lightning, I know for sure I would want to give the MATLAB users far less funding, or even none at all, since they're really going to struggle to properly leverage the CPUs and GPUs available to them, whereas the Lightning people will largely just have this work immediately, and almost certainly be able to iterate faster and be more likely to find something meaningful.

It's a contrived and unfair example, and in practice the real problem is actually the annoying MATLAB licensing mostly, but also it really is a fact that MATLAB screws up even basic stuff in HPC environments (see e.g. https://docs.alliancecan.ca/wiki/MATLAB#Simultaneous_paralle...).

[esalman]

> If both request significant compute allocations, and I see one is planning on MATLAB, and the other on PyTorch Lightning, I know for sure I would want to give the MATLAB users far less funding, or even none at all

I want to say the least offensive way possible that I really do hope you do not ever get to review NSF/NIH grants because of bias like this.

The example that you gave is/was also a solved problem in the HPC systems I used. SLURM/PBS, when properly configured, can ensure that local clusters do not spill over and not use resources that are not supposed to be used.

[D-Machine]

I picked an intentionally contrived case where MATLAB is undeniably a really bad choice here. Most real world situations are not so ridiculous and would not justify such a simple decision, but if you can't see that MATLAB wouldn't be a horrible choice in this specific contrived situation, I would suggest it is in fact you who has the bias. I agree in general anyone making funding decisions solely based on MATLAB vs. other is not to be trusted, but that is NOT the situation I so carefully presented.

And yes, clearly, based on what I linked, this MATLAB issue is a solved problem on Canadian SLURM HPC clusters too. It's just, so much more is solved by e.g. modern Python libs like Ray, Lighting, and etc., it is really silly to pretend that MATLAB users can as easily make as good use of HPC clusters as everyone else can, especially today if GPUs are involved.

I'm still going to upvote you here because I think you are engaging honestly and fairly, and we just generally disagree here.