The manifold is in the middle (“small input space is expanded onto a big manifold and contracted again”) so f(manifold) would need to be in the middle too.
The problem really is one has to find out what constitutes the embedding and output if you want to apply f(manifold) but as others have pointed out that’s out of scope for this paper. My only insight here is that it is not surprising at all that one layer, or even a single function pass, suffices to get the desired reinforcement. I don’t actually know if there’s any research that quantifies the “manifold threshold” of each layer to try out this approach but it would be interesting for sure