[TehCorwiz]

I used Claude to build a complete data extraction pipeline for a popular current best seller book series: audiobook -> text (via whisper) -> local LLM (qwen) -> database. Not once did it seem to acknowledge or care about copyright. It even used knowledge it already had about the books to exclude certain ones before beginning since the character I was interested in did not appear in those. It definitely had context of what we were working on.

[js2]

Why would you go from audiobook to text? Is there no epub available?

[TehCorwiz]

Not without DRM. It was easier to buy the audiobooks and use the analog loophole to get text. It's probably less accurate, but for what I'm doing it was fine. Names were the worst, but whisper at least made the same mistake each time so a simple search+replace handled most of the obvious edge cases.

[4gotunameagain]

If you are going to be illegal, you might as well use library genesis and get DRM free ebooks :)

[awakeasleep]

On the other hand, it’s a beautiful example of the abilities LLMs have bestowed upon us, where it’s easier for a guy to transcribe audiobooks then to use a website to quickly download an epub

[TehCorwiz]

To your point about time, from beginning of the project to transcripts in markdown tagged with extra metadata was about 3 hours. That's LLM planning, building whisper.cpp twice and running ROCm vs Vulkan benchmarks, testing whisper and adjusting prompts to handle edge cases, then processing the books.

Most of the books weren't available on lib gen or Anna's Archive. The few I did find were themselves obviously transcripts. Easy tell was they were missing distinctive formatting that I knew existed from reading the dead tree edition. At that point it was easier to make my own. I probably spent an hour searching for eBooks without DRM that weren't transcripts. Do they exist somewhere? Probably, but with a search of unknown length it was a better use of my time to make my own transcripts with what I had on hand.

I was really wanting to make commentary on how chaotic LLMs are even under constrained circumstances. No doubt both system prompts includes language about considering copyrights and trademarks. Probably pretty strong language at that. For whatever reason one LLM didn't "feel" like translating a 1000 year old document but another did not care in the slightest that we were ripping text from new audiobooks.

[js2]

Huh, in my experience audiobooks are more expensive and/or harder to find than ebooks. Out of curiosity, can you give an example of one such audiobook you transcribed?