Hacker News new | ask | show | jobs
by ilaksh 1018 days ago
Respect for persistence.

For the screenshot thing, are you using the GPT-4 visual understanding? I assume not since it's only available for a few groups.

Looks like an interesting project.

Are you using BLIP-2? I understand if you don't want to give away secrets. Just thought it couldn't hurt to ask.
1 comments
jarodxu 1018 days ago
Thanks for the support!

We only used the standard GPT-4 API (without visual capabilities).

I think it's okay to have a some discussion here. We employed traditional OCR, GPT-4, and some of our own algorithms to assist GPT-4 in understanding the context and relationship within the image scene (allow me to retain a bit of mystery here).

link
bradknowles 1016 days ago
What about the privacy concerns mentioned above?

Are you doing all this work locally on the device, or are you shipping everything to OpenAI servers, or maybe other servers in some cloud?

If you're doing all processing locally on the device, I might be interested. But I don't want all that data being shipped off to some cloud for remote processing there.

link
jarodxu 1015 days ago
In our current web app, users' screenshots are stored on AWS cloud. Only the essential information processed from these screenshots is sent to the OpenAI API. According to OpenAI API's public commitment, this data will not be used for any training purposes.

For our upcoming Mac client and iPhone app, we're considering storing the screenshots on the device. However, we still need to weigh the pros and cons of cross-device syncing versus local storage. Your feedback is invaluable in guiding our final decisions for the Mac client and iPhone app. Thanks for sharing :)

link
bradknowles 1015 days ago
When you've got a local-only solution, or where local-only is at least available as a configuration option, please let me know.

Thanks!

link
jarodxu 1015 days ago
Absolutely! We'll keep you posted once a local-only option is available. Thanks for your interest!

By the way, are you more excited for the iPhone app or the Mac client?

link
bradknowles 1015 days ago
I don't generally take screenshots on my iPhone, only my Mac. But I'm sure a lot of people would like to have this app on iOS, too.
link