[xg15]

This is really cool and I hope there will be more experiments like this.

My takeaway is also that we don't really have a good intuition yet how the internal representations of neuronal networks "work" or what kind of internal representations can even be learned through SGD+backpropagation. (And also how those representations depend on the architecture)

Like in this case, where the author first imagined the network would learn a logic network, but the end result was more like an analog circuit.

It's possible to construct the "binary adder" network the author imagined "from scratch" by handpicking the weights. But the question would be interesting if it could also be learned or if SGD would always produce an "analog" solution like this one.