[bjornsing]

As I understand it a randomized controlled trial (RCT) can determine causation. But I don’t think “what they did with this setup” was an RCT. A real RCT in this context should IMHO assign new cases randomly to a “treatment” or “control” group. In this setup they assign legal precedents randomly, but all new cases are effectively under “treatment”. That’s not the same thing.

[asdff]

How are new cases effectively under treatment? They identified 150 decisions, wrote articles for each, and assigned half randomly to treatment or control. they did not consider newer cases beyond this set of 150. An RCT is a type of randomized experiment, but all determine causation:

https://en.wikipedia.org/wiki/Randomized_experiment

[bjornsing]

You’re right. It’s the legal precedents / decisions that are the “patients” here. The new cases (that cite those precedents / decisions) are just… let’s say “test results”.

So yes, this is a way to determine causation in relation to what legal precedents / decisions will be cited in new cases depending on the contents of Wikipedia.

I was under the (perhaps mistaken) impression that they claimed a causal link between their Wikipedia articles and the outcome of new cases. The chosen setup cannot IMHO prove such a causal relationship, in the strict statistical sense. That would require randomizing new cases and applying the “treatment” (new Wikipedia articles) only to some of them.