[Sargos]

This only works if the developer uses app indexing where Google can search through the content and make it searchable to end users and other endpoints. It doesn't come for free so Google has no access to things like your bank app's screen.

[wikyd]

http://arstechnica.com/gadgets/2015/05/android-ms-google-now... makes it seem like they just walk the view hierarchy. They show examples of using this in a chat app, which makes less sense to be found via app indexing.

[zmmmmm]

The presenter actually mentioned app indexing on stage at some point so I think it does rely on the developer opting into indexing.

On the other hand, it's not clear to me that the user opts into the indexing, so Google might suddenly silently be getting access to a huge number of user's deep within-app data without them realising it. But that is old news, app indexing has been around for a while.

[wikyd]

I think the way to think about it is Google is doing what it does best: search. They use whatever content they can scrape from the screen using the view hierarchy once the user signals intent that they want to search. It's a faster way to do search using context (in this case the screen) to provide the "keywords". The app indexing part is how they are able to show related apps with deep links. They use app indexing of other apps to find relevant results for the query.

App indexing makes sense when you have more stable content inside of an app like a traditional public website. Think Yelp and their 'screen' per place inside of the app. It doesn't make sense to publish the contents of a chat conversation to Google's app index -- that would be a huge privacy leak! It would be the equivalent of GoogleBot indexing my Gmail inbox.

[Roritharr]

Do you have ans links or sources for that? I couldnt find anything so far.