[w-m]

It's amazing to see deep learning blast through all the benchmarks, for example in computer vision, over the last couple of years. At the same time something starts to feel off about having all these single-use asymmetric feedforward networks solving their own little task. Being trained in one direction, then used in the other, then thrown away. Maybe being chained together for a more complex task, but that seems to be about it for the average (real-world application) use case of deep learning nets.

I'm sure there's plenty of interesting work being done in ML to improve on this situation and come up with new architectures. Yet I was moderately surprised when I rediscovered Boltzmann machines recently, and found not much work seemed to be going on there at all (very little at NIPS 2017 for example?).

This BEAM seems intriguing, here's hoping it opens the door to a better understanding and modeling of our world.

[nafizh]

RBMs went out of fashion after 2010-2011, as other architectures worked better than them in almost all of the tasks in vision.

[rm_-rf_slash]

I have had a similar thought recently. The office I work at has a large e-recycling bin for old computers. I have recovered quite a few desktops, laptops, and monitors, as well as a bunch of tidbits like adapters and RAM.

A lot of the RAM, for instance, is DDR2 and usually a measly 1gb apiece. They take up the exact same amount of space as RAM with 4gb apiece or more. I don’t know entirely why I still have them. Now that I’m doing physical computing/IoT development, Im seeing how pointless it is to have a bunch of desktops/laptops when I can get much more done - conveniently I might add - with a teeny tiny RedBear microcontroller.

I think an inherent feature of technology is having to get used to the idea that things age and die much faster than other products. Whether that’s physical hardware or trained neural networks, there comes a point when we just have to let go.

[radarsat1]

I found this one pretty interesting in that regard. The basic gist is that learning the function that projects onto the boundary of the dataset is useful for a variety of (linear inverse) problems.

One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models https://arxiv.org/abs/1703.09912