[beyondCritics]

Wow, reading this am suddenly noticing that hiding the latency in system code can be much more simpler than i thought it is. Say i have a function

  X f(X u);

on which i want to iterate occasionally to get in turn f(u0),f(f(u0)),... The "smart" way to do this, is

  X u=f(u0);     //Initialize once
  ...
  X smart_f() {
    X w = u;
    u = f(w);
    return w;   // This line is not stalled by the previous one, 
                // hence a super scalar processor might be able to hide the latency of calculation f(w)    
  }

I doubt any compiler will be able to figure this out, and surely not if f has side effects.