[krisoft]

> eg. having some samples (not all) checked by another lab,

I don't think that is useful at all in case of rare diseases. You would just get two reports saying that the random sample is free of HIV.

Much better would be to send some known control samples. Making sure that some of the samples is known HIV+, and then check if the supplier can tell which ones are those.

[kelnos]

You can still do this kind of audit, but you need to test a statistically significant number of samples in your "spot check" such that you know you some of them will be infected. The number will vary depending on the incidence of a particular type of infection present, but this is data that should be available.

I agree that sending control samples can also be effective, though. But if you need to send the whole organ to the test lab (and not just a small tissue sample), you probably don't want to be wasting healthy organs by infecting them. Better to just wait until you have an organ that's known to be infected already.

[krisoft]

> But if you need to send the whole organ to the test lab

Why would you need to do that? Realistically the sample needed here is a small vial of blood from the organ donor’s body.

> You can still do this kind of audit, but you need to test a statistically significant number of samples in your "spot check" such that you know you some of them will be infected.

Nah. It really doesn’t work. The problem is that HIV is very rare. (HIV incidence per 1000 population adults 15-49 in Brazil is between 0.34 - 0.45[1])

Let’s be ultra conservative and set the “spot check” rate at 100%. That is you are sending samples from every single body to two labs. Because of the low incidence rates you would still expect hundreds and hundreds of those samples to return as negative from both labs. This might work if you would somehow have a “gold standard” lab you trust and an other “less trusted lab”. But in reality there is no such a thing as a “gold standard” lab you can trust without QA. (And if there would be you would just use them, instead of the other lab.) Even with that ridiculously high “spot check ratio” you wouldn’t know if you are getting negative results because they are in fact negative, or because both of your labs are falling for some reason and giving you constant false negatives.

In conclusion spot checking the results with a second lab simply doesn’t work. Even if you spot check every single organ donor you would be still blind for even the most basic error cases for unacceptably long times.

On the other hand if you intermingle a control sample into every single batch that changes the game. Lets say they run the tests on batches of 10 and you make sure that a random one of those is always known to be positive. Now if something goes wrong and they don’t detect the sample you can straight away reject the whole batch of tests as faulty. And it only costs you an 11% extra over not doing any QA.

So with the “spot checking” test you can pay as much as 100% extra and still not know if the tests are having the most elementary kind of fault for hundreds and hundreds of organ donors. Or you can go with the “control sample” strategy and have a reasonably high confidence for every batch right away at much less of a cost. Yeah you can do the “spot check” audits but it is ridiculously bad at catching issues even if you spend a lot of money on it.

1: https://www.unaids.org/en/regionscountries/countries/brazil

[DoctorOetker]

I agree with you, also the bogus argument of "since most people are HIV free..." assumes direct testing instead of pooled testing (using modern information theoretic optimized pooled testing).

A bit of data is most informative if the entropy is 1 bit as well. A signal that is true most of the time, or a different signal that is false most of the time is less informative. Use pooled testing such that the result is true or false half of the time.

Had information theoretically justified pooled testing been applied from the start, then:

* 1) control-testing the testing contractors would have been straightforward and passing 10 control samples by chance would have a likelihood of 1 over 1024.

* 2) it would have made obvious that saving money on control-testing the contractors would hardly save any money

* 3) even in the bad scenario that control testing was skipped, the issue of contractors cheating would have surfaced much faster, since combining the pooled tests to identify which patient tests positive would constantly result in mysteries, meaning control-testing needs to be enabled, not the mathematics of pooled testing brought in doubt.

* 4) testing pharma industry hates pooled testing, as it means technological competition instead of sales growth by abusing the naive but false "common sense" that you need as many tests as patients tested.

on a side note: assuming tests with different operating point on the RoC curves (having different false positive vs false negative ratios) have different prices, do we know if the operators blatantly provided fabricated results, or if they blatantly ignored basic mathematics and thought the more expensive tests could be substituted by the cheaper ones even if intended for a different purpose?

consider a test designed for telling a patient that we diagnosed HIV, and then consider a test designed for screening an organ to be inserted into a patient.

do you think they should both use the same test? or do you think it wiser to have the diagnosis test have lower false positive rates, and the organ screening test to have lower false negative rates?