[dotinvoke]

I built a service that turns entire websites into structured output: https://sitewideai.com

You enter a starting URL, describe the data you want in a prompt, the AI suggests columns for the output spreadsheet which you can customize, and then goes off and turns the website into structured data into a CSV file.

It also supports limits, you can say for example "visit at most 100 pages" and it will stop after 100 pages.

It was easier said than done to get prompts working as intended and the crawler to focus on the most relevant URLs first. As always, the final 20% end up taking up 80% of the development time.