This doesn't necessarily have to be at the UI level. If sites have a well defined API they can still be indexed but serviced for users by native apps. In fact, this is better than kludgy scraping.
For that you'd have to have a standard API with indicators of content importance. Even then it would be akin to keyword meta tags which are bad representations of site content in general.