Patent examiners can look basically anywhere a member of the public can and some other places.
The internal search tools check a lot of patent databases and are quite good, but take time to learn.
The USPTO has a lot of subscription databases as well, including fancy AI/ML-based ones.
Many examiners will also search normal search engines like Google, though this can be tricky for legal reasons. If the application was not published yet then examiners are not allowed to get very specific in the search and other search engines as that could release confidential information to the search engine. The USPTO has agreements with the subscription databases to keep the searches confidential but no such agreement exists with Google.
In my view making new search tools like https://www.priorartarchive.org/ would not help the situation too much. It would be better to integrate more databases into the existing USPTO tools, as they are designed for serious power-searchers, and would make the new databases more visible. The internal search tool is much faster than the alternatives and operates by keyboard. Point-and-click search is much slower by its nature. Speed really is critical when time constrained and I think that is something not appreciated outside of patent organizations.
What I'm reading in your description is that the process is designed to find cases where someone has patented the same thing before, but not designed to find things that should be fundamentally unpatentable ('obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art') and many other people are doing the similar thing in their products but not trying to patent it (because it shouldn't be patentable) and not explicitly writing it up in public blog posts with the exact same keywords.
Presuming that "ways to wirelessly share photos online such as through social media" has not been patented before, the best place to look for prior art would be the actual existing social media products which have ways to wirelessly share photos online instead of historical patent data; an effective search would have to be for actual prior art (i.e. products and solutions), not descriptions of prior art (patents and webpages).
Why is the secrecy required if the patent is going to end up public anyway? If you could pre-publish to the public (with a verifiable timestamp) then anyone could submit a challenge and the patent would be only be granted if it’s actually novel and remains unchallenged for some time. IDK I must be missing something because this seems too obvious a solution...
Not all US patent applications will be published. The applicant can pay extra to keep it secret unless it is granted.
As for why the applications aren't published as soon as possible, I'm not sure. I suspect people might want some time before publication to test the market. If there is no market demand, I know that some law firms will abandon the application. But if it's worthless, then why not publish it?
The internal search tools check a lot of patent databases and are quite good, but take time to learn.
The USPTO has a lot of subscription databases as well, including fancy AI/ML-based ones.
Many examiners will also search normal search engines like Google, though this can be tricky for legal reasons. If the application was not published yet then examiners are not allowed to get very specific in the search and other search engines as that could release confidential information to the search engine. The USPTO has agreements with the subscription databases to keep the searches confidential but no such agreement exists with Google.
In my view making new search tools like https://www.priorartarchive.org/ would not help the situation too much. It would be better to integrate more databases into the existing USPTO tools, as they are designed for serious power-searchers, and would make the new databases more visible. The internal search tool is much faster than the alternatives and operates by keyboard. Point-and-click search is much slower by its nature. Speed really is critical when time constrained and I think that is something not appreciated outside of patent organizations.