[samstave]

Yay! I love Shot Scrapeer - I wish you had made it a decade ago!

Thanks for shot scraper.

Off the top of you head what would be the easiest command to have shotscraper barf a directory of shot-scraper HTMLs each day from my daily browsing history.

This would be interesting if I have a browsing session for learning something and I am researching across a bunch of sites - roll it all up into a Digi-ography of the sites used in learning that topic?

---

I've always been baffled that this isnt an inate functionality in any app/OS - its a damn computeer - I should have a great ability to recall what it displays and what you have been doing.

Heck - we need our machines to write us a daily status report for what we did at the end of each day.

Surely that would change productivity. If you were force to do a self-digital-confession and stare you ADHD and procrastination right in the face.

[simonw]

Yeah, things like Archive Box are probably a better bet there. But... you could write a script that queries the SQLite database of your history, figures out the pages you visited then loops through and runs `shot-scraper html ... > ...html` against each one.

I just wasted a few minutes trying to get Claude 3 Opus to write me a script - nearly got there but Firefox had my SQLite database locked and I lost interest. My conversation so far is at: https://gist.github.com/simonw/9f20a02f35f7a129b9850988117c0...

[eichin]

My "cheat" for "poke at chrome's sqlite database for current live state" is that they're always locked but none of them are that big, just make a copy and query the copy. `current-chrome-url` runs in `real 0m0.057s` and does a `jq -r .profile.last_used ~/.config/google-chrome/Local\ State` to get the profile, then copies `~/.config/google-chrome/"$PROFILE"` into a mktemp dir, then `sqlite3 -cmd 'select url from urls where id = (select url from visits where visit_time = (select max(visit_time) from visits));' $H .exit` on that copy.

[genewitch]

This used to be fairly simple to do before https everywhere, just install squid (or whatever) and cron the cache folder to a zip file once a day or whatever.

There's paid solutions that kinda do what you want, but they capture all text on your screen and OCR it to make it searchable, which at least lets you backtrack and has the added advantage that it will make pdfs, meme images, etc searchable, too. last i heard it was mac only but a few folks mentioned some windows software that does it too.

as an aside i don't consider reading/learning nearly all day to be a net negative, even if ADD is to blame. (i haven't had the "H" since i was a child.) A status report wouldn't "stare" me in the face; in fact, it would be nice to have some language model take the daily report and over time suggest other things to read or possible contradictions to link to.

[jimmySixDOF]

look at ArchiveBox from the comments below

[Hendrikto]

> Heck - we need our machines to write us a daily status report for what we did at the end of each day.

I am sure Trump, Xi, Putin, etc. would like that very much.