[Bug] Setting mixing_weights=False in SoftmaxLikelihood still adds the learnable parameter W

[DiegoFreitasH]

🐛 Bug

In the SoftmaxLikelihood docs it is said that setting mixing_weights=False sets $\mathbf{W} = \mathbf{I}$, however this is not that case.

I belive the problem is the following line from SoftmaxLikihood code, where mixing_weights=False still makes the condition True.

if mixing_weights is not None:
            if num_features is None:
                raise ValueError("num_features is required with mixing weights")
            self.num_features: int = num_features
            self.register_parameter(
                name="mixing_weights",
                parameter=torch.nn.Parameter(torch.randn(num_classes, num_features).div_(num_features)),
            )
            if mixing_weights_prior is not None:
                self.register_prior("mixing_weights_prior", mixing_weights_prior, "mixing_weights")

To reproduce

** Code snippet to reproduce **

import gpytorch

likelihood = gpytorch.likelihoods.SoftmaxLikelihood(num_classes=10, num_features=10, mixing_weights=False)

print(list(likelihood.parameters()))

** Output **

[Parameter containing:
tensor([[-0.0793, -0.1665,  0.1279,  0.2622,  0.0725,  0.1549, -0.0300, -0.0283,
         -0.0591,  0.0089],
        [ 0.0322,  0.0715,  0.0669, -0.0969,  0.2169, -0.1513, -0.0649,  0.0333,
          0.2200,  0.1891],
        [-0.1031, -0.0766, -0.1974,  0.0179, -0.0828,  0.0129, -0.0505, -0.1171,
         -0.0514, -0.0053],
        [ 0.0614,  0.0770,  0.1645, -0.0909, -0.0493, -0.1035, -0.2445, -0.0129,
          0.0691, -0.1129],
        [-0.0502, -0.2440,  0.0376, -0.1998,  0.0396, -0.1529, -0.0346, -0.2143,
          0.0672, -0.0409],
        [ 0.0320,  0.1339, -0.0355,  0.1170, -0.1188,  0.2840,  0.0124,  0.0653,
          0.0609, -0.0100],
        [ 0.1042, -0.0188, -0.0342,  0.1988,  0.0767, -0.0106,  0.1875,  0.0822,
          0.1039, -0.0689],
        [-0.0150,  0.1847,  0.0074,  0.1407, -0.0876,  0.0497, -0.1405,  0.2031,
          0.1750, -0.0365],
        [ 0.0145,  0.0502, -0.2817,  0.2650,  0.0530,  0.1424,  0.1745,  0.1005,
          0.1670, -0.1164],
        [-0.1080,  0.0253,  0.0779,  0.0927,  0.0031, -0.0185,  0.2311,  0.0065,
          0.0173,  0.1895]], requires_grad=True)]

Expected Behavior

The script should print an empty list, because there should be no learnable parameters.

System information

Please complete the following information:

• GPyTorch Version (1.11)
• PyTorch Version (2.4.1+cu121)
• Ubuntu 24.04.1 LTS

Additional context

It should be noted that setting mixing_weights=None has the expected behavior, where $\mathbf{W}$ is not a learnable parameter.

[gpleiss]

Ah yes... that line should probably be if mixing_weights:, not if mixing_weights is not None.
Could you submit a PR, and add an appropriate unit test?