[SPACECADET3D]

Author of Symbolica here: Symbolica is used in physics calculations to do arithmetic on rational polynomials that are hundreds of megabytes long.

For my physics research I have worked with expressions that was just shy of a terabyte long and had > 100M terms. The way that works is that you stream terms from disk, perform manipulations on them and write them to disk again. Using a mergesort, terms that add up can be identified by sorting them to be adjacent.

[glimshe]

This is fascinating. What is the real-world application of these polynomials? I mean, what technical-scientific problems can only be solved with such large objects? I love the thought of "if I don't solve this enormous polynomial, this solar panel won't have very good efficiency" or something like that.

[SPACECADET3D]

These polynomials appear when computing Feynman diagrams, which are used to make predictions for the Large Hadron Collider. The collider can measure collisions with such astonishing precision (<1% error) that predictions of the same order are also needed. The more precise you want to be, the more terms in a series approximation of the mathematical description of the collision you need to compute. For example, I computed the fifth-order approximation of the QCD beta function, which governs how intense the strong force affects matter. This takes 5 days of symbolic manipulations (pattern matching, substitutions, rational polynomial arithmetic, etc) on 32 cores.

The large polynomials appear in the middle of the computation, often referred to as intermediate expression swell. This also happens when you do a Gaussian elimination or compute greatest common divisors: the final result will be small, but intermediately, the expressions can get large.

[eh_why_not]

Is there a document/book you can recommend that includes a simplified example/tutorial of this computation process from beginning to end? (Or, what are the right keywords to search for such a thing?)

I'm looking for something like: here's the particle interaction we will work on, this is a very simple Feynman diagram, and here's the simplified data the LHC gave us about it, here's the resulting equation from which we'll derive a series, etc.

Not looking for how to program it, but actually for seeing the problem structure, and the solution design from beginning to end. (Familiar with high level physics concepts, and comfortable with any math).

[Y_Y]

I like Peskin's Introduction to QFT, but Zee's QFT in a Nutshell is also good.

[amelius]

You can even make a toy-problem with large polynomials yourself.

Just make a Taylor approximation of some e.g. transcendental function like sin(x) around some point, and use more and more terms to get a higher precision.

[lav42]

I used polynomials and specifically special symmetric polynomials extensively during my Phd research (mathematical physics).

For instance symmetric polynomials (x_1^2 + x_2^2 + ...) can describe the state of a system of particles where exchanging any 2 particles does not change the system at all (exchange x_1 and x_2 in the previous expression, same polynomial = same system & state).

If you have a system of equation that you can solve exactly with special polynomials, you can approximate real world system governed by similar equations by using your special polynomials as a starting point and adding a correction to your solution.

There's so much to say about polynomials, but I'll leave you with a basic example that shows how multiplying infinite polynomials allow you to count the number of ways there are to hand you back your change at the till:

The basic units of change are 0.01$, 0.05$, 0.10$, 0.25$, 1$, 5$, 10$, ... For example, you can always give exact change back with only 0.01$.

So the set of all the different amount of change you can produce with 0.01$ is given by the exponents in the following

sum_n>=0 (q^(0.01))^n = q^0 + q^0.01 + q^0.02 + ... q^348.47 + ...

Now we can get all the amounts you can generate with 5 cents:

sum_k>=0 (q^(0.05))^k = q^0 + q^0.05 + q^0.10 + .... and so on for 25cents, 1$, ...

Notice now that given the multiplication properties of polynomials, that multiplying:

(sum_n (q^0.01)^n) * (sum_k (q^0.05)^k) * (sum_l (q^0.10)^l) Will give you all the different amounts you can generate with 0.01, 0.05 and 0.10.

For instance with n=5, k=1, l=0 you get 0.10$

q^(0.01 ^ 5) * q^(0.05 * 1)

You can get 0.10$ with n=10

q^(0.01 * 10)

You can get 0.10$ with l=1

q^(0.10 * 1)

Finally you can get 0.10$ with k=2 q^(0.05 * 2)

So when you multiply

(sum_n (q^0.01)^n) * (sum_k (q^0.05)^k) * (sum_l (q^0.10)^l)

altogether, you get

1 + ... + q^(0.01 * 5) * q^(0.05 * 1) + q^(0.01 * 10) + q^(0.10 * 1) + q^(0.05 * 2) + ... = ... + 4 q ^ (0.10)

There are thus 4 ways of handing back exactly 10 cents.

So for any amount, you take the following: product(c in (0.01, 0.05, 0.10, 0.25,...) (sum_n (q^c)^n) = sum_(a >= 0.01) [Number Of Way To Give Back Change for amount `a`] * q^(a)

So that would be the "generating series" of the number of ways to hand back change.

In this context, polynomials bridge the gaps between combinatorics and analytical computation.

[anthk]

That an exercise on SICP, a Scheme learning book. And there are similar ones for Common Lisp.

[lav42]

Say what? I'm always astounded that people think everyone knows all the acronyms for whatever field they're working in.

[masfuerte]

https://en.wikipedia.org/wiki/Structure_and_Interpretation_o...

[anthk]

By just googling 'SICP Scheme' the hints are pretty clear...

[lav42]

The book looks awesome, thanks for pointing out, I'll give it a read.

[temporarara]

While I agree with this attitude towards obscure acronyms, SICP is widely known among those who voluntary enter a site called Hacker News, and it's easily searchable too.

[actionfromafar]

I love how merge-sort which was once used on punch cards because RAM was tight is still relevant. :)

[lanstin]

Knuth's section on sorting data on tape drives is surprisingly relevant to big data running over S3 buckets and streaming into compute.

[ThomasBHickey]

I can remember the days! I suppose as problems grow in size, it's not too surprising that older methods of coping with what seemed like lots of data are still applicable.