Hacker News new | ask | show | jobs
by im3w1l 2001 days ago
Wouldn't it be easier to first do your crawling and then construct the matrix in one go? But graphs more broadly sounds like an area where you might want to update the matrix yeah.
1 comments

Well, one exception could be that some parts of the web need to be crawled more often than others.

But tbh, I think incremental updating of a matrix only makes sense if the other parts of your pipeline are also incrementalized.