Hacker News new | ask | show | jobs
by vessenes 72 days ago
A few things about AI-led projects like this come to my mind — first, it’s cool to see all this pulled together. I’m sure the design will read “Claude 2026” soon, but that’s fine - it’s clean and generally has reasonable UX.

There are some real rough spots - for instance, the Latin texts are generated via OCR from scanned documents directly; they’re not from some other scholarly corpus that’s been checked. I only looked at a few, but they all have significant transcription difficulties. Sources are linked, and those sources seem to be archive.org scans. Of course, getting a fluid-sounding translation out of a somewhat shitty transcription is something AI will do for you happily, but it’s harder to get it to tell you where it’s gone off the rails.

That’s not the thing that comes to mind, though. What comes to mind is that projects like this are super useful scaffolding, and I hope it’s built as such. Transcription will get better. Actually I’m pretty sure it could be better now, given the output quality. Translations of better transcriptions will be better. Plus we will likely have higher quality translation tech available.

So, I’d like to see a project like this lean in to that iterative side of this kind of scholarship/hobby/historical work and make versioning and logging of updates part of the interface. Starting in the late 1990s many academic projects did this with large corpuses of documents, (I’m familiar at the least with the Yale Jonathan Edwards project), and used crowd sourced support — there’s no reason not to include facilities that interleave the AI and interested Latin/Roman scholars here.

In my mind with that done, this could turn into a genuinely useful tool. Which would be cool!

2 comments

I haven't checked any texts from the 500s. But I did some work with texts from the 1700s. Most of them had terrible transcriptions on archive.org, made using old tesseract versions. You could probably improve a lot with newer tesseract versions. I went for the nuclear option and just passed the image of each page (along with some context on how the previous page ended) to Qwen2.5vl:32b and got near-perfect transcriptions. And as you can tell by the old model that was months ago, vision models only got better.

Of course in some cases vision models are a liability for OCR because the errors they do make are replaced by plausible sounding replacements instead of alphabet soup. But if you only use the transcription as input for an LLM that doesn't matter. It only becomes an issue of how much compute you are willing to throw at it

Yes, exactly. What could be durable is not the specific transcription as of today - until it’s perfect or at least ‘good enough’ - but the web site, comments, and process that can be run and turn into improved results - that part seems likely to be valuable to me.
Thank you so much for the feedback on this. I just implemented some updates that should better track our iterations and changes to the project. Obviously, GitHub does some tracking, but I've added some more formal updates.

Additionally, I completely agree on the OCR issues. Long-term, the goal is to use higher quality OCR and a broader set of data to make this even more accessible to people.

I'll note that the primary goal is as much scholarship as it is giving access to hobbyists or people interested in the data itself. If the data could get to the point where it is scholarly useful, of course that would be something I'd like to achieve.