[pitt1980]

"To sidestep the randomness of observed outcomes, we focus on modeling the probability of a successful outcome for a given case. Secondly, to account for the context or difficulty of the case, we measure the change in the probability of a successful litigation outcome, from substituting a legal service provider in. The difficulty of the case is not important, but rather the impact a legal service provider has on that original probability of success. From now on, we will refer to this measure as the impact on expected performance. This substitution exercise is known as a counterfactual, as we apply the substitutions to historical data."

-----------------

how exactly did they model the probability of successful outcomes for given cases?

because those depend on highly detailed fact patterns, many of which are largely qualitative in nature

if they were actually able to put all those in a data base that you could then model

that'd be a more interesting accomplishment to read about

[Digory]

Right. The post links to slideshow "model" for counterfactuals given at Hulu. And I won't pretend to analyze it scientifically, but it's hard to shake the feeling that it's circular in this application.

We want to rank lawyers based on effect on outcome, not win-loss record. But it looks like the model is one that derives "counterfactual" outcomes by making comparisons based on win-loss records.

Can someone give a better lay-level discussion of the expected outcome model?