[monstertruck]

(I work at System)

First, thanks!

By internal contradictions, do you mean conflicting evidence in the relationship between topics or metrics? That will (and does) come up regularly - peer-reviewed studies investigating the same topics have differently measured (or contradictory) results. We have tools for assessing the statistical quality of submitted relationships (through things like statistical reproducibility, algorithm type, statistical controls, etc.), so unreproducible or statistically unlikely relationships will be clearly seen as such. Building tools to programmatically test reproducibility of evidence is definitely something we've thought about (if that's the "formal verification" you are talking about).

Ultimately the goal will be to (statistically) approximate the sum total of all evidence between pairs of topics, and also to provide users with the tools and sources to assess (and apply!) that evidence.

[derbOac]

I guess I had a few reactions offhand:

1. Pretty interesting.

2. The criteria for reproducibility seem a little rough to me? They seem to be sort of distant from things like registered replication, publication bias analysis etc.

3. My gut impression is you need some kind of meta-scientific model, eg something that models the probability of studying an association, the observed association conditional on that, effects on heterogeneity of effect sizes etc.

4. Along those lines, I wonder if there's an implicit schema of looking for nonzero associations and documenting them rather than reporting best estimated strength of known association? Maybe not.

5. I'm curious how you define nodes/topics versus subnodes/subtopics. I suspect defining the nodes/topics and their boundaries would become tricky?