[TaylorAlexander]

Well it’s not midi but what you’re describing is similar to this approach:

https://magenta.tensorflow.org/ddsp