[datadrivenangel]

"Combining the computer hardware speed increase of 4,000 times with the solver software performance improvement of 5 million times, the total improvement from 1989 to 2024 is a factor of 20 billion times faster!"

No wonder people have started making jokes that programmers no longer know how to program! We can get away with a lot more now.

[dragontamer]

The software improvements are on an order of 1000x more than the hardware improvements.

IE: The software matters more. As in, today's programmers know more about this subject.

[theLiminator]

I'd say on average programmers know less about writing optimal code now, but certainly the best in the field know a lot more.

[owlbite]

I'd say that the algorithm matters more. As in, today's mathematicians know more about the subject.

I suspect the actual implementation of said algorithms probably achieves a lower % of peak performance than the older ones (though to be fair they are /much/ more complex algorithms).

[Mr_Minderbinder]

>I suspect the actual implementation of said algorithms probably achieves a lower % of peak performance than the older ones

In my experience I have found the opposite to be the case. Most old maths libraries are written in FORTRAN (generally an order of magnitude or more slower than a comparable C/C++) and the implementations of standard algorithms are often sub-optimal and naive. I got the same impression when I compared arctangent (Taylor series) implementations in π programs and Minimax in chess (see Bernstein's program for the 704). I would guess that in the worst case they were 100x slower than what those machines could theoretically achieve. The C+inline asm libraries of today might be 2-10x slower at worst and some are even bottlenecked by memory. In this case I doubt the programs discussed in the 1989 paper, which were written in Prolog and FORTRAN, are exceptions to this.

[wiredfool]

I went to a Guest lecture in ~96 on performance of supercomputing and applications to FEA, so basically matrix factoring.

In the time from the Cray 1 -> then, there were 6 orders of magnitude of hardware gains, and 6 orders of magnitude in software as well.

[nwallin]

Matrix factoring as in LU, Cholesky, QR, SVD etc? 6 orders of magnitude from mid-70s to mid-90s?

Unless I'm misunderstanding I'm shocked that there was that much left on the table.

[wiredfool]

I think it went from naïve gaussian through LU and SVD to approximate iterative forms for the top eigenvectors/values. So a good portion of that was not computing the higher order terms that didn't significantly contribute to the results.

Hazy memory though, as it was 25 years back and I've been out of the FEA side of things for 20+ years now.

I will say though -- I was doing some stuff at the time that was burying SuperSparcs for 24 hours at a time, and would now probably run realtime on a watch or phone. (Again, a big mix of hardware advancement, reduced precision for insignificant terms, and generally optimized algos)

[owlbite]

FEAs probably involve sparse matrices, which have a lot more complexity than simple dense matrices. For example compute optimal reordering of a generic sparse matrix is iirc NP-complete.

[taeric]

Of course, it takes knowing how to program to take advantage of the solver improvements. I find that most programs do not, in fact, use any solver techniques. More, I think I can program, but I would also have trouble using solvers in a lot of my programs.

[hinkley]

The problem with factorials is that 20 billion doesn't necessarily mean much.

Let's say we could solve a problem of 100 elements 35 years ago, with 100! operations. If the 2x10^10 multiplier only made the exact same calculation but faster, that would let you solve n = 105 in the same time. Business problems don't grow that slowly. Not over 35 years.

You have to get better at solving the problem in less than n! steps by culling impossible scenarios, and doing it more aggressively.