My experience on iOS: scrolling feels wrong because it is non-native (different inertia and velocity), reader mode stops working, and if for whatever reason I want to share original link that won't be easy.
Because the AMP pages are often functionally crippled compared to the real site and require clicks to get off and on to the real web page (see Reddit, The Guardian etc), as well as the negatives of giving Google that level of control.