[tachyonbeam]

IMO, what these adversarial examples give us is a way to boost training data. We should augment training datasets with adversarial examples, or use adversarial training methods. The resulting networks would only be more robust as a result.

As for self-driving cars, this is a good argument for having multiple sensing modalities in addition to visual, such as radar/lidar/sonar, and multiple cameras, infrared in addition to visible light.

[andbberger]

But at what point do you have to wonder if we're using the wrong basis? And how do you know that augmenting the data with tiny adversarial perturbations won't just leave the network vulnerable in a different direction?

It's pretty obvious how to build translational symmetry into a net that's still expressive and easy to train (convolution). But you have to spoon feed CNNs rotational and other symmetries by augmenting the training data. What you really want is a model that has all the symmetries your data has built in.

My sense is that the community at large seems to regard DL as a magic blackbox which it really is not. Complete basis of function + finite data = guarantee of wonky interpolation between samples. What you really need to do is restrict the class of expressible functions to those you need - build your prior into the model.

[azag0]

This is a huge topic in applying ML in physics and chemistry where we already have a lot of prior detailed knwoledge about the systems we want to describe and it would be silly not to build it into the ML models.

[yakult]

What's the current state of art in this direction? Is there a way to encode equations explicitly prior to training?

[azag0]

People now try to use ML anywhere and everywhere so it's wild west a little. Three examples: [1] uses a standard neural net to represent a many-body wave function, with all the machinery of quantum mechanics on top of that, and reinforcement learning to find the true ground state. [2] uses a handcrafted neural net, which by construction already takes advantage of a lot of prior knowledge, to directly predict molecular energies. [3] uses a simple kernel ridge regression coupled with a sophisticated handcrafted scheme to automatically construct a good basis (set of features) for a given input, to predict molecular energies.

In all these cases, the ML itself is not the target problem, but only a tool, and most effort goes into figuring out where exactly to use ML as a part of a larger problem, and how to encode prior knowledge, either via feature construction or neural net handcrafting.

[1] http://sci-hub.io/10.1126/science.aag2302

[2] http://sci-hub.io/10.1038/ncomms13890

[3] https://arxiv.org/pdf/1707.04146.pdf

[andbberger]

Variational inference

[DonbunEf7]

You are, unfortunately, probably just playing out Mr. Crab's obsession with record players.

Remember that these tricky images are based on the principle that machine-learning algorithms are differentiable and high-dimensional. There is a lot of ways to transition between, say, the desktop dimension and the cat dimension, and it's all continuous, so we're guaranteed to be able to influence the machine in that sort of direction.

You could imagine somehow taking all of the adversarial examples and categorically augmenting a machine's learning to know about the examples, creating a cat-masquerading-as-desktop dimension. But all you've done is make a lot more space (by adding a dimension) and so the next iteration of adversarial examples will be able to proceed by the same process as before, just on this new augmented machine.

[yorwba]

But we don't really care about the cat-masquerading-as-desktop category in itself, so an adversarial example that makes a cat look like a cat-masquerading-as-desktop, or masquerades a cat-masquerading-as-desktop as a cat, isn't really relevant.

By adding enough adversarial examples to the training set, you can absolutely immunize a model against adversarial perturbations of the training data.

The problem is that the volume of "not very different" data points surrounding an example grows exponentially with the input dimension, so you need to train for much longer, and your "adversarial protection" will likely overfit to the neighborhood of training examples, which doesn't help with unseen data.

[mannykannot]

We care about the existence of a nontrivial set of images that demonstrate a troubling lack of robustness in image classifiers, at least until we we have good reason to say with confidence that such failures will not be a problem in practice.

[kevinnk]

> IMO, what these adversarial examples give us is a way to boost training data.

That's basically the idea behind GANs.