[PaulHoule]

No off-the shelf information extraction system is going to be useful for your task. In particular most of the systems you list are notorious rabbit holes and dead ends. (Well, UIMA was developed by IBM to support projects that have 100+ coders and data entry people, it's not a dead end if you have a budget that big...)

If getting the right answer matters for you you need to start with a workflow system that will let you do the task manually. You will absolutely need it for two reasons: (1) editing cases that the extraction system gets wrong, (2) creating a training/evaluation set for the extraction pipeline.

When you've got a well-defined task you can do manually then you can think about automating some of the extraction (80% is realistic) with rules like regexes, RNN/CNN/Transformer models.

My contacts in Argentina who do projects like this all the time say that it takes maybe 20,000 examples to train an extraction model and that fits my experience. What separates the people who succeed at this kind of project from those who fail is that those who succeed make the training set, those who fail exhaust themself looking at projects like Tika, OpenNLP, UIMA, etc.

[gpa]

Sounds reasonable. And yes, I am a one-man team. But then, after collecting all this information, how do you organize and store all this information for on-demand availability? For tables extracted from the publication, you can create a new table withing a relational database. But how do you organize single data points extracted from the text? Where do you store them?

[PaulHoule]

If I were writing a workflow system in a hurry I'd use arangodb in the back end and a web server based on asyncio. I like the idea of something RDF-based but I'd need to develop the right algebra for isolating individual "records" in an RDF database so they can be updated safely.

I worked at a startup that built a system with quite a few parts including a system for annotating text to train an extraction pipeline. That last bit had a typescript/react front end, a scala web server, and kept data in an elaborate set of tables in postgres. They were in a hurry to get it working for one particular customer so it wound up pretty half-baked.

I have a lot of ideas on this so look up my profile and send me an email!