> My fascination with these sequences began in 1964 when I was a graduate student at Cornell University in Ithaca, NY, studying neural networks. I had encountered a sequence of numbers, 1,8,78,944,13800,..., and I badly needed a formula for the n-th term, in order to determine the rate of growth of the terms (this would indicate how long the activity in this very simple neural network would persist). I will say more about this sequence in Section 2.1.
It’s really fascinating to bump into mentions of NNs from the 60s & 70s. They seems to be quite hot at the time. The paper on the Medial Axis Transform mentions neural networks too, in a way that makes it seem like it was the cool thing to do. By the time I was in college, NNs were very out of fashion.
Here’s the NN problem Neil was working on, and the first sequence in the database: https://oeis.org/A000435
Yea neural networks were actually invented in the 40s by Warren McCulloch and Walter Pitts at University of Illinois at Chicago. They had a few isolated results until GPUs and distributed computation really kicked them into high gear and that made the change in terms to “deep learning” and now GPT-3 and other networks are hyperparamaterized neural networks with millions to billions of parameters .
I was part of a research group that extensively trained small neural networks for image-processing in 2001, the high-energy physics community had been using them for many years by that time.
Furthermore, I believe that the PalmPilot's handwriting-recognition engine also had a neural-network component.
Agreed that the usage has increased radically in the last twenty years, but even before the GPU-based revolution, it felt like neural networks were already broadly known and in use across the sciences and engineering. They were just slower :).
True, but scaling has its own problems. It was necessary to find better optimisers, activation functions, regularisers, weight sharing schemes, architectures and many other ingredients to make it work. And to prepare the large datasets, and invent the whole stack of frameworks, from CUDA to HuggingFace.
We have had 250,000 ML papers written since 2012. That's a lower bound on the number of distinct experiments necessary to find the winning tickets of today. Inventing the step-activated neuron formula was less than 1% of the way here.
Yes, I’m also glad to hear it’s future path is already paved. Sloane described the process and history in the paper:
“In 2009, in order to ensure the long-term future of the database, I set up a non-profit foundation, The OEIS Foundation Inc., a 501(c)(3) Public Charity, whose purpose is to own, maintain and raise funds to support The On-Line Encyclopedia of Integer Se- quences or OEIS.
On October 26, 2009, I transferred the intellectual property of The On-Line Ency- clopedia of Integer Sequences to the Foundation. A new OEIS with multiple editors was launched on November 11, 2010.
Since then it has been possible for anyone in the world to propose a new sequence or an update to an existing sequence. To do this, users must first register, and then submissions are reviewed by the editors before they become a permanent part of the OEIS. Technically the OEIS is now a “moderated wiki”.
I started writing this article on November 11, 2022, noting that this marked twelve years of successful operation of the online OEIS, and also that the database is in its 59th year of existence.”
The one thing I wish is they had a keyword for base-ten related sequences (rather than only "base" for any base), because base ten related sequences simply are almost always going to be way more recreational maths related than base two or base three related sequences.
>>> import heapq, itertools
>>> seq = heapq.merge((2**i for i in itertools.count(1, 2)),
(9*2**i for i in itertools.count(1, 2)))
>>> list(itertools.islice(seq, 0, 4))
[2, 8, 18, 32]
import itertools
def sigma(n):
return sum(i for i in range(1, n//2+1) if n % i == 0)
def seq():
for n in itertools.count(1):
if sigma(2*n) % 2 == 0: continue
if sigma(3*n) % 3 == 0: yield n
continue
>>> list(itertools.islice(seq(), 0, 4))
[2, 8, 18, 32]
The others are either degenerate for 4 points of the A001105 case, or are not so easy to implement in raw Python.
This makes me so happy to read. I had the privilege of working on the same lab as Neil (and Dave Applegate, another notable person in OEIS). No exaggeration at all to call them geniuses, you hang out with them for 5 minutes and know they're cut from different cloth. Nicest folk, too.
>"My fascination with these sequences began in 1964 when I was a graduate student at Cornell University in Ithaca, NY, studying neural networks. I had encountered a sequence of numbers, 1, 8, 78, 944, 13800, . . ., and I badly needed a formula for the n-th term, in order to determine the rate of growth of the terms..."
Related Mathologer video:
Mathologer - "Why don't they teach Newton's calculus of 'What comes next?'"
The finite difference method of that video is only useful for finding polynomial sequences. Of course, any finite sequence can be extended to some polynomial, but in many cases (such as this one) that’s not the result you’re looking for.
> Robert Jackson suggests that if you've completed a difference table and still don't understand the sequence, you should turn the paper through an angle of 60 degrees, say, and start again and perhaps repeat this several times to make a fan of difference tables.
Because this sequences isn't polynomial. It's https://oeis.org/A000435 , with the explicit formula
a(n) = (n-1)! * Sum_{k=0..n-2} n^k/k!
and the approximate form shows it's grows roughly as n^n:
a(n) ~ sqrt(Pi/2)*n^(n-1/2)
Here's my Python implementation:
from math import factorial
from fractions import Fraction as F
def A000435(n):
return int(factorial(n-1) *
sum(F(n**k, factorial(k)) for k in range(0, n-1)))
OK, I think I understand what you and anderskaseorg mean by polynomial/non-polynomial sequences...
If we think about a polynomial, say 3x^2 + 2x + 1 -- then that's basically an algorithm that says "take x, raise it to the second power, muliply it by 3, take the result of that, add it to x multiplied by 2, and then take the result of that, and add one to it".
In other words, in that algorithmic definition,
a) There is no recursion
(note that factorials imply recursion in an algorithm -- even though they could be computed by using a simple look-up table)
and,
b) There is no division
(which could result in non-integer values)
So, in the formulas you give, you are using both recursion and division to form your sequences.
OK, so number tables / triangles / fans (call them what you will) -- don't work for things like that.
I am willing to buy into that, prima facie, but "with the proverbial grain of salt"...
You see, there's something deeper about math -- that we're not understanding here...
To understand what it is or may be (I don't know what it is, all that follows is mathematically speculative reasoning, and might be wrong, might be quite wrong indeed!), then I would suggest the following:
Because this is the simplest (AFAIK) recurrence relationship (AKA recursive, "defined using recursion") integer sequence -- that can be produced.
To recap, its definition is:
F(n) = F(n-1) + F(n-2)
(with F(0) = 0 and F(1) = 1)
Now let's create that integer sequence -- and a corresponding number table / difference table / triangle / fan (again, call it what you will) -- and let's see if that works...
Now, I don't have Python all set up to do this -- all I have is pen and paper.
But I tried it -- and lo and behold, it works!
What's very interesting about the Fibonnaci Sequence -- is that if you create a number table for it -- you'll see that it repeats (although each row is shifted to the right!) in descending rows!
In other words, that number table -- if we can spot that pattern -- is in fact showing us the recurrence/recursive relationship!
In other words, it's still working(!) -- for this simple recurrence/recursive formula!
But we know that it fails -- somewhere between this simple recurrence algorithm -- and the one you have presented!
My challenge to you then, as a fellow Mathematician (I haven't done this by the way, I'm lazy! <g>) -- is to figure out when/where/why the number table / difference table / triangle / fan -- fails -- between the simplest of all recurrence relationship formula, the Fibonnaci sequence -- and this one!
Because you see, I'll bet there's some interesting mathematical knowledge there!.
I'd do it myself -- but no time!
Besides, you have Python already set up and running and everything... I don't!
Anyway, I think it would be interesting to know this!
Also -- once the exact failure criteria are understood -- next question is, is it possible to construct an n-dimensional table (like maybe 2 or more interlinked/interrelated number/difference tables) -- where one maps to others, and you can get the correct answer for deeply recursive algorithms -- which include division?
I don't think you're thinking about these concepts in a useful way.
Yes, factorials can be defined through recursion. But so can multiplication of non-negative integers:
def mult(a, b):
if a == 0 or b == 0: return 0
if a == 1: return b
return mult(a-1, b) + b
>>> mult(5, 9)
45
Furthermore, factorials can alternatively be defined through the Gamma function, which supports more than just the integers and isn't defined recursively. https://en.wikipedia.org/wiki/Gamma_function .
> the simplest (AFAIK) recurrence relationship ...that can be produced.
The positive integers is even easier. For n >= 0:
F(0) = 0
F(n) = 1 + F(n-1)
> What's very interesting about the Fibonnaci Sequence
Another interesting thing about the Fibonnaci Sequence is that it has a closed-form solution, given in your OEIS link as:
It is also non-polynomial. In the limit, it approaches a simple exponential.
Nor is it bounded by a polynomial, meaning that if you pick any positive integer p and any constant C, and define g(n) = C n^p then for a large enough n you will find that |g(n)| < |F(n)| .
Furthermore, A000435 , with its n^n - like form, grow even faster than exponential. It's similar to factorial growth.
> as a fellow Mathematician
I am not a mathematician. Neither are you.
> s to figure out when/where/why the number table / difference table / triangle / fan -- fails -- between the simplest of all recurrence relationship formula, the Fibonnaci sequence -- and this one
Consider f(n) = f(n-1) + 1 where f(0) = 0. This is the sequence 0, 1, 2, 3, .... This addition of 1 can be seen trivially in the difference table.
Consider f(n) = f(n-1) + f(n-1) with f(1) = 1. This is the sequence 1, 2, 4, 8, 16, 32, or 2^n, which is an exponential function.
Consider the Fibonacci function f(n) = f(n-1) + f(n-2) with f(1) = f(2) = 1. This uses an addition of the two previous numbers. It approaches an exponential function as n gets larger.
Consider f(n) = n * f(n-1) with f(1) = 1 This is the factorial. It grows faster than the exponential function.
Because the last one uses a multiplication instead of addition, a difference table (which is based on subtraction) won't show the pattern. The inverse of multiplication is division, so use a division table.
> deeply recursive algorithms
These are not deeply recursive. For an example of those, see the Ackermann function. https://en.wikipedia.org/wiki/Ackermann_function , which is not primitive recursive like functions we've discussed so far.
For real numbers, there’s the dictionary of real numbers (https://www.amazon.com/Dictionary-Real-Numbers-Jonathan-Borw...), “a list of just over 100,000 eight-digit real numbers in the interval [0,1) that arise as the first eight digits of special values of familiar functions”
My favorite hard core nerd insult used to be "Your idea of a hot date is looking up dirty words in the unabridged dictionary," but now I'm going to use "Your idea of a hot date is looking up 69 in the Handbook of Integer Sequences."
It’s really fascinating to bump into mentions of NNs from the 60s & 70s. They seems to be quite hot at the time. The paper on the Medial Axis Transform mentions neural networks too, in a way that makes it seem like it was the cool thing to do. By the time I was in college, NNs were very out of fashion.
Here’s the NN problem Neil was working on, and the first sequence in the database: https://oeis.org/A000435