[osolo]

My team worked on this feature and I can confirm that yes, semantic search is being done for both text and images. This allows you to do fuzzy searches like you mentioned in your comment and use words to match images. Everything runs on your device (all the models, the vector database, etc.) in order to preserve privacy.

[simonw]

That's really impressive. Can you share which embedding models you are using for this? Also, is Phi-3 or Phi-3 Vision involved?

[osolo]

I don't think they've announced publicly what models we're using. I don't think there's any particular reason for this, but just in case, I can't name them here. I'll see if this can be addressed in a blog post of something.

I can tell you that Recall isn't using Phi. Rather, it's using a collection of models that are much more tuned (and therefore much more efficient) for the feature.