Question regarding heteroskedastic noise

[Duane321]

Hey GPyTorch team,

Hope you are all getting a lot out of GPyTorch - I know I certainly am!

My question regards kernel design: I'm hoping to increase the variance of a few test data points without changing my point predictions for those points too much. From looking through the available kernels, I can't see any obvious way to make this behavior happen.

I'll provide my best guess. Let's say I have some kernel defined and I'd like to change it to have this behavior. I could apply a multiplication by a linear kernel that resolves to 1 whenever the data points isn't one of those special test points. Otherwise, it's some large number (maybe 2). So far I haven't had much luck with this, but I think I could just be screwing up the optimization. This also does impact the point prediction, but that might be unavoidable. I'm hoping that the variance parameter on the linear kernel gives me my increase in the prediction interval around this test point.

Do you think I'm on the right path? Do you have recommendations?

Any words are appreciated,
DJ

[jacobrgardner]

One way to accomplish this might be to use a FixedNoiseGaussianLikelihood:

gpytorch/gpytorch/likelihoods/gaussian_likelihood.py

Lines 102 to 110 in 5b0425f

	Example:
	>>> train_x = torch.randn(55, 2)
	>>> noises = torch.ones(55) * 0.01
	>>> likelihood = FixedNoiseGaussianLikelihood(noise=noises, learn_additional_noise=True)
	>>> pred_y = likelihood(gp_model(train_x))
	>>>
	>>> test_x = torch.randn(21, 2)
	>>> test_noises = torch.ones(21) * 0.02
	>>> pred_y = likelihood(gp_model(test_x), noise=test_noises)

In the last line of this example, a supplied vector of noises are added to the test predictions. This won't affect the point predictions at all, because it effectively amounts to inflating the variance of the test points, while the test means remain unchanged.

Is this roughly what you are looking for, or am I misunderstanding something?

[Duane321]

oh this is perfect! I didn't realize that FixedNoiseGaussianLikelihood allows for different noises for different test points.

As always, thanks for the help Jacob

[jacobrgardner]

No problem! FixedNoiseGaussianLikelihood is pretty new, and we have so much documentation to catch up on we don't have example notebooks for that yet, just the docstrings demonstrating it.

[eytan]

@jacobrgardner, @Balandat, I wonder if the name FixedNoiseGaussianLikelihood is a little misleading, as FixedNoise (at least on the surface) sounds like a homoskedastic noise model. Perhaps something like KnownNoise would be a little less subject to this misinterpretation?

[Balandat]

sounds reasonable to me, I'll let @jacobrgardner decide if he wants to make that change.