| > The commenters in this thread are writing off the data because...? They decided the measure is bad? Yes, because what the measure actually measures isn't a valid proxy for what it purports to measure. > When the measure conforms to experience, it's probably worthwhile to look into it. No, if the adopted proxy (here, "LOC per commit") has some sound rationale for being used as a proxy for the actual quality of interest (here "expressiveness"), then it is worth actually getting some results with it for which you have a firm expectation of what those results would look like if you were able to directly measure the quantity (in this case "expressiveness") for which you are using the proxy (in this case "LOC per commit"). If after such testing the proxy -- which you first looked to for reasonableness, and then tested on the "simple" data for which you had a firm expectation of what the results would be for the quality of interest -- seems workable, its worth investigating what kinds of results in returns for things which you don't have a firm idea of where they would fall. (Which is the only reason you actually use a proxy measure for in the first place.) In this case, the proxy fails at the first test (sound rationale for using it as a proxy for expressiveness), which makes the second test (do the results line up with what you'd expect on a known sample set) meaningless. |