[Docs] Passing Custom 2D mean function & Variance AND previously trained model

[tniggs84]

📚 Passing Custom 2D Mean Function & Variance AND Previously Trainer Model

Hello, this is a follow-up to my question raised on Issue #674 that was addressed by @jacobrgardner (thank you so much by the way!)

I really appreciate your comments and they sound potentially like just what I need. The problem is that I still am unclear how I could actually pass this in as a mean function. I have looked at the documentation and the earlier thread and I don’t understand how this would get passed in.

For instance, in a 1-D case, what if I had the following vector:

mean_vals = [100, 30, 25, 40, 18, 10, 5, 2, 1, 1]

What would you do to pass this through?

Thank you again in advance. I’ll be playing with the fixed noise vector in the mean time!

Ted

[gpleiss]

@tniggs84 - I read back through #674. I'm not entirely sure this is what you're asking about, but if I understand correctly: you have a strong prior belief that the means at x1, x2, ... xn is equal to [100, 30, 25, 40, 18, 10, 5, 2, 1, 1]. What you want now is a mean function that will generalize to other points besides x1, ..., xn. Is that correct?

Going on the assumption that this is true: I would suggest that you just stick with a constant or zero mean function.

The challenge that you have identified is - how do I extend the mean to generalize to data points other than x1, ..., xn - and the problem is that there is no good solution to this. There are many possible solutions, such as using kernel interpolation (but then you'd have to choose the appropriate kernel/hyperparameters), a neural network (but then you'd have to choose an architecture), or some custom polynomial shape (again, you'd have to choose a bunch of hyperparameters). Basically, creating a mean function like the one you describe will require a lot of design decisions/hyperparameters that will take a lot of manual tuning and requires a lot of prior knowledge about the problem. There is no one-size-fits-all solution.

On the other hand, if you do something simple like a constant mean or a zero mean - the Gaussian process should still fit the data rather well. And this way you have a lot fewer parameters/design decisions to make.

[tniggs84]

@gpleiss thank you for your reply. I am effectively looking at the following situation. I have a series of trials and these trials tend to follow a very similar mean shape (at least in one axis). If we restrict this in 1D, then the array I listed could represent the mean shape I tend to see and my thought was that i could use prior training data to estimate noise around these observations. Without any additional information, this would reflect my generic shape. My next thought was that when actual values started to arrive, I could collapse the noise around these observations and then reprocess the model.

Is this possible? If so, how would I do that? I’d also be happy with training a GP on the training data and then using that as the prior mean and covariance matrix, but I would still need to collapse all noise around actual sad they arrive.

[gpleiss]

Sorry for the very slow reply - this got lost in my notifications.

I'm not sure what you mean by "collapsing the noise"? Have you tried training a zero-mean GP on this data? Even if the mean does have a very similar shape (which could possibly be exploited), designing a good mean function can be quite difficult. It is much simpler to just let a zero-mean GP automatically discover this structure.