[dharma1]

I mean, it's a cool project, but I'm not sure the latent space between statistically learned sound representations is that interesting for music... For speech synthesis I can see the applications.

For musical sound generation I think there is still room for new discovery with physically modeling sounds, like these guys are doing - http://www.ness.music.ed.ac.uk/. You can morph between physically modeled instrument sounds too, and it's all realtime. Complex physical models tend to have too many parameters for human performers to control so maybe neural networks could be used there to learn to control the params and their interaction in a meaningful way for musical sound production.

Also, since WaveNet is computationally so intensive, training and re-constructing sounds sample by sample, at 16k per second at 8bits turns out to be pretty heavy going - let alone 44khz/16bit. So to make it realtime this implementation is basically interpolating in a grid of pre-rendered wavetables to morph between instruments, right?

[MrBuddyCasino]

I think Yamaha made a physics based synthesizer. Only a few were produced because they were expensive and hard to master. Can't remember the name though.