[samwalrus]

I like the notion of 'context' for transfer learning. Where context can be parameterized.

The idea is that you learn a general model from your available data and you are able to specialise that model to perform well when adapted to different contexts.

A simple example is when the different contexts are different cost matrices or different expected ratios of positives and negatives.

So instead of learning a classification model from your training data, you learn a ranking model. You can make and adapt the different classification models (thresholds on the ranking) depending on the context of where that model will be deployed.

So for example you learn a ranking model from pictures that ranks women above men. When you want a classifier that classifies pictures as men or women, you chose the threshold from your ranking model depending on the confusion matrices costs for the context of where the model is being deployed.

I think a cool research theme is to think of similar tools for other aspects of transfer learning.