UserWarning: You set iterator_valid__shuffle=True after upgrading to 0.12.0, but it is False

[rhjohnstone]

After upgrading to 0.12.0, I get the warning before training starts:

UserWarning: You set iterator_valid__shuffle=True; this is most likely not what you want because the values returned by predict and predict_proba will be shuffled.

However, I definitely create the model with iterator_valid__shuffle = False. And, just to confirm, after training I do

print(estimator.get_params()["iterator_valid__shuffle"])  # False

I'm hoping the warning itself is wrong, otherwise I don't see how I can fix this within my own code.

[rhjohnstone]

I'm pretty sure this is a Skorch bug in 0.12.0.

In an earlier version, this parameter was checked here:

        # warn about usage of iterator_valid__shuffle=True, since this
        # is almost certainly not what the user wants
        if kwargs.get('iterator_valid__shuffle'):
            warnings.warn(
                "You set iterator_valid__shuffle=True; this is most likely not "
                "what you want because the values returned by predict and "
                "predict_proba will be shuffled.",
                UserWarning)

so it's actually looking up the value of iterator_valid__shuffle.

However, in the new version, it is done here:

        # warn about usage of iterator_valid__shuffle=True, since this
        # is almost certainly not what the user wants
        if 'iterator_valid__shuffle' in self._params_to_validate:
            warnings.warn(
                "You set iterator_valid__shuffle=True; this is most likely not "
                "what you want because the values returned by predict and "
                "predict_proba will be shuffled.",
                UserWarning)

so it's only checking for the existence of iterator_valid__shuffle as a kwarg when creating the estimator, not checking its value.

[BenjaminBossan]

Thank you for reporting this, it's indeed a bug, showing a false positive. In your case, could you just not set the value to False, as it's the default anyway?

[BenjaminBossan]

@rhjohnstone I created a bugfix PR (#908). As this issue is quite minor, we probably won't make a patch release of skorch, I hope that's okay.

[rhjohnstone]

@BenjaminBossan thanks! No problem about a patch release; I really just wanted to confirm that what I thought was indeed happening.