[ogrisel]

But neither declarative DL toolkits (theano & tensorflow) use that C++ language feature: the computation graph is typically defined by writing a python script to assembles building blocks dynamically at runtime.

Once the graph is defined, it can be passed along with concrete values for the input nodes to the runtime framework to execute the section of the graph of interest (possibly with code generation + compilation).