[osdf]

Not to argue with you, robrenaud, but Hinton himself writes in their 2006 paper 'A Fast Learning Algorithm for Deep Belief Nets':

The greedy algorithm bears some resemblance to boosting in its repeated use of the same “weak” learner, but instead of reweighting each data vector to ensure that the next step learns something new, it re- represents it.

I guess that most people however would not think of this interpretation of greedily pretraining deep networks :). (I wonder if mbq had this in mind).

In the same article your point about good models of the input is mentioned, too (only copy&paste a small part of the paragraph):

Unsupervised methods, however, can use very large unlabeled data sets, and each case may be very high-dimensional, thus providing many bits of constraint on a generative model.

The 2006 paper is really an amazing read in my opinion.