Hacker News new | ask | show | jobs
by lrei 3292 days ago
Afaik there is no meta data scrapped (technically I think some HTML meta tags are scrapped but not sure if used).

There is machine-learning in the annotations - categories rely on a (cross lingual) text classifier, entities rely on matching to Wikipedia articles, maybe a bunch of small other things to - I don't know all the details - there are some papers published about it.
1 comments
oceanbreeze83 3292 days ago
interested in knowing about these papers. can you point me to some?
link
pbadenski 3292 days ago
A query to Google Scholar and quick skimming through the papers yields this as interesting start:

Event Registry – Learning About World Events From News http://wwwconference.org/proceedings/www2014/companion/p107....

Using news articles for real-time cross-lingual event detection and filtering http://ai2-s2-pdfs.s3.amazonaws.com/f917/c0cff24fed1af45f94c...

link
lrei 3292 days ago
Correct. Main author of the project is Gregor Leban:

https://scholar.google.co.uk/citations?user=5pAxBWsAAAAJ&hl=...

The original crawler (newsfeed.ijs.si) paper is from

Trampus, Mitja and Novak, Blaz: The Internals Of An Aggregated Web News Feed. Proceedings of 15th Multiconference on Information Society 2012 (IS-2012).

link