[zhengyi13]

It's not precisely your desire/use-case, but that sounds vaguely like you might like some of the programmatic sound generation systems out there:

Overtone - http://overtone.github.io/ Tidal - https://tidalcycles.org/

Perhaps those might be hooked into something you're already building?

[stevehiehn]

Thanks! I'm aware of super collider and it is cool. However i'm not interested in real-time. I have been generating midi patterns and rendering audio server-side over many hours. I've started hand rolling a system to play drum samples from generated patterns. It would be nice if i could find something already built.

[adamsea]

Maybe a music-oriented programming language like Chuck, Supercollider, or Pd?

I would recommend looking at Pd (Pure Data), which, imho, is the open source version of Max/MSP, a very popular graphical programming language used for music composition as well as the creation of audio applications.