Transform test data

[RocaVincent]

Hi,

If I understood correctly, we pass our target variables as argument of the fit method (discrete_covariates or continuous_covariates). But it seems that we also have to give this variables for the transform method. But it's not correct to give variables to transform that we want to predict, no ?

[molly00ecla]

The combat was a harmonization method, instead of a model. The input and output of the method were both data, so the discrete_covariates or continuous_covariates should also be given.

[rafaloz]

Following this question up, although I understand that combat is a method and not a model, the method uses the parameters learned in training and uses in tests. So in this way, I guess there wouldn't be data leakage. However, if you are using the method to predict any of your covariates you will need the covariate for the test transformation, and in that case, my guess is that you are leaking. Is this correct?

[sahahn]

Following this question up, although I understand that combat is a method and not a model, the method uses the parameters learned in training and uses in tests. So in this way, I guess there wouldn't be data leakage. However, if you are using the method to predict any of your covariates you will need the covariate for the test transformation, and in that case, my guess is that you are leaking. Is this correct?

I just found this implementation today, and have only looked at it briefly, but I believe your understanding of the situation is correct. The way they have the code setup is that calling transform on new subjects would be fine (as long as they are from seen sites). That said, as you mentioned, if one of your covariates was also the variable you are predicting, then you would want to be careful as you should not pass this transformer object the real target value, or as you noted, you will introduce data leakage.

So I actually also recently wrote a sklearn compliant version of Combat (https://github.com/sahahn/neuroCombat/blob/master/neuroCombat/combat.py), and the way I handled this issue was with a include_y attribute, which if set to True, would include your target during training / fit_transform, but then during transform on new subjects will essentially just use the mean of the true target value as a feature. So using this code, the way to replicate that would be just when calling transform on new subjects you would pass the mean of the target variable on the training set as the value (instead of the true target value which you are not supposed to know)