[jeeceebees]

I think the interesting part would be example based audio synthesis. Could you replace a synthesizer with a neural network which, when fed examples, would allow you to generate sounds / explore some latent space between the examples.

For example an approach similar to https://gauthamzz.github.io/2017/09/23/AudioStyleTransfer/ but then using the methods described in the PixelNN paper.

[radarsat1]

I'll just plug my recent work on my sound synthesis "copier": http://gitlab.com/sinclairs/sounderfeit

It more or less attempts to be what you describe. Not very polished yet, but I had some basic success in modeling the parameter space of a synth, and adding new latent spaces with regularization.