[jellyksong]

Two questions:

1. I wonder how many times the test set can be used on "incremental changes in future versions of the model" before losing statistical validity.

2. This article describes their process, but not the FDA's process. Are there specific regulatory requirements for ML models beyond their four types of reports?

[yshrestha]

1) AFIAK there are no hard and set rules for this. I think this would have to be the manufacturer's judgement call. Good point though, with enough time you may end up just over-fitting to the test set.

2) The FDA's guidance on ML models is still in flux. Please see https://www.fda.gov/medical-devices/software-medical-device-...