[ogrisel]

No it means writing a program that defines the structure of a computation graph lazily (without executing the nodes when defining the model) so as to reuse that compute graph in a later step of the programs.

The computation graph is an in-memory datastructure that can be introspected by the program itself at runtime so as to do symbolic operations (e.g. compute the gradient of one node in the graph with respect to any ancestor input node).

theano implements this in pure Python and can generate C or CUDA code from string templates (in Python). tensorflow has to a Python API to assemble pre-built operators which are mainly written in C++ and use the Eigen linear algebra library.

[billconan]

"defines structure of a computation graph lazily (without executing the nodes when defining the model)"

But this sounds exactly like expression template.

[ogrisel]

But neither declarative DL toolkits (theano & tensorflow) use that C++ language feature: the computation graph is typically defined by writing a python script to assembles building blocks dynamically at runtime.

Once the graph is defined, it can be passed along with concrete values for the input nodes to the runtime framework to execute the section of the graph of interest (possibly with code generation + compilation).