Heterogeneous Multi-output Gaussian Process

[michaelcao28]

Hi all,

New to gpytorch here. Just wondering if there has been any implementation on constructing a heterogeneous multi-output GP using approximate inference? what I mean by this, the likelihoods associated with each output/task are not the same and non-Gaussian. Task_1 could follow a Poisson distribution, and Task_2 could follow a Bernoulli. Both Task_1 and Task_2 are correlated in their latent parameter.

Any suggestions on how to implement this if no such code is available?

[gpleiss]

This is probably best done using the low-level pyro inferface: https://docs.gpytorch.ai/en/v1.6.0/examples/07_Pyro_Integration/Pyro_GPyTorch_Low_Level.html

We don't currently have any examples of a composite likelihood (not one that we've published, anyways), but this should allow you to have a heterogeneous likelihood. Here's how to update this example:

1. Wrap the variational strategy with a LMCVariationalStrategy (see this example), which will output MultitaskMultivariateNormals rather than single-task distributions.
2. The function_samples term in the model method will be of size num_samples x ... x N x num_tasks (rather than num_samples x ... x N in the single task example). You can then split this term into multiple pieces to get your Bernoulli likelihood inputs, Poisson likelihood inputs, etc.
3. At the end of the model method, sample from each of your Poisson/Bernoulli/etc. observation models. You'll replace the code after it says "# sample from observed distributions"

Sorry we don't have a better example available right now, but hope this helps!

[michaelcao28]

thanks @gpleiss ! Sounds very doable. I'll have a go at implementing this soon :)