[vaughngh]

I have tensors in later layers whose shapes depend on the training of the previous layers.

Rad! Do you have any examples (or literature) that explains when this is beneficial?

[gormanc]

Not yet! I'm not using convnets or backprop or anything so I don't think it would be beneficial that way, but you could get something similar to what I'm doing by looking at Fritzke's Growing Neural Gas[1]

[1] http://papers.nips.cc/paper/893-a-growing-neural-gas-network...

[vaughngh]

Neat, thanks for the link.