All that AMP does for speed is impose limits on CSS and JS size (plus givs access to Google CDNs). You can simply write small assets without the need for Google to specify this.
I've recently built an AMP version of a marketing website (since AdWords supports AMP cache, thought it would be useful for speed). The truth is, while AMP as a library does provide solid speed improvements on its own by implementing all the best practices, the real speed improvements is the cache with preloading - as well as the other stuff it does like image optimisation.