[Bug] settings.variational_cholesky_jitter does not work in run time

[LuhuanWu]

🐛 Bug

To reproduce

Using settings.variational_cholesky_jitter in the model run time does not change the actual jitter value

** Code snippet to reproduce **

from gpytorch.models import ApproximateGP
from gpytorch.variational import CholeskyVariationalDistribution
from gpytorch.variational import VariationalStrategy
from gpytorch import settings

class GPModel(ApproximateGP):
    def __init__(self, inducing_points):
        variational_distribution = CholeskyVariationalDistribution(inducing_points.size(0))
        variational_strategy = VariationalStrategy(self, inducing_points, variational_distribution, learn_inducing_locations=True)
        super(GPModel, self).__init__(variational_strategy)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

train_x = torch.randn(10)
inducing_points = train_x
model = GPModel(inducing_points=inducing_points)

with settings.variational_cholesky_jitter(float_value=1e-2):
    print(model.variational_strategy.jitter_val)

** Stack trace/error message **

The printed output is 0.0001, which is the default jitter value.

Expected Behavior

Expected value is 1e-2.

Additional context

In fact, the jitter value can be set through settings when instantiating the model:

with settings.variational_cholesky_jitter(float_value=1e-2):
    model = GPModel(inducing_points=inducing_points)
    print(model.variational_strategy.jitter_val)

The printed output is 1e-2, which is expected. However, such way of setting variational_cholesky_jitter value is not consistent with other settings, e.g. settings.cholesky_jitter.

[gpleiss]

Agreed. It should be possible to set this value dynamically, rather than when the model is initialized.

Additionally, it doesn't really make sense to have a bunch of different context managers for jitter values - we should think about consolidating them.

I'll put up a PR soon.