| Thank you!!! It's so funny that you mentioned "scriptable notebooks like Jupyter or Mathematica" since I have been spending all of my time on mathinspector recently. I think our points of view are actually very strongly aligned. However I believe the next big idea is likely to come from outside of computer science. Personally, I am betting on biology. So many of the most sophisticated techniques are based on biology, e.g. neural nets and genetic algorithms. I have done a lot of work on extending the theory of computation with a new axiom which gives Turing machines a self replicating axiom[1], [2] In many parts of science, there is a cross pollination, where new ways of thinking about subject X come from a new discovery in subject Y. Typically, research will follow a group think pattern until it hits a brick wall, then you need that really big breakthrough idea. This line of reasoning leads to the conclusion, imo, that it's approximately equally likely to come from either pure computer science, or pure mathematics, or somewhere else. [1] https://math.stackexchange.com/questions/3605352/what-is-the... [2] https://medium.com/swlh/self-replicating-computer-programs-8... |
Alan Turing invented neural nets in a little-known paper entitled Intelligent Machinery (see [1]), in 1948. Since, the use of NNs has moved away decisively from inspiration by nature. I reckon, nature's last big win in AI were convolutional NNs: Kunihiko Fukushima's neocognitron was published in 1980, and inspired by 1950s work of Hubel and Wiesel [2]. Modern deep learning is largely an exercise in distributed systems: how can you feed stacks and stacks of tensor-cores and TPUs with floating point numbers, while minimising data movement (the real bottleneck of all computing)?
Not unlike, I think, how airplanes were originally inspired by birds, but nowadays the two have mostly parted ways, for solid technical reasons.
[1] http://www.alanturing.net/turing_archive/pages/Reference%20A...
[2] https://en.wikipedia.org/wiki/Neocognitron