[Question] Multitask Hadamard with batched variational inference

[irinaespejo]

Hello team 👋

I've looked around the documentation examples and issues to implement a Multitask GP Regression like this one but the dataset is pretty large so I would like to also implement SVGR like in this example.

I have checked this other example but my main problem is that the input data is irregular. Meaning that for every input point I only have one evaluation of one task.
I have also checked other issues like #1743 , #1433 , #1285 and #657 but found no conclusive answer (I am a bit lost with all these issues)

More precisely I have:

• Task 1: number of points n_1 and data (x_1, y_1) with x_1 in R^4, y_1in R
• Task 2: number of points n_2 and data (x_2, y_2) with x_2 in R^4, y_1in R
• y_1 and y_2 are correlated (and it is necessary to exploit this in order to get a good fit)

Does GPyTorch support this type of model in a way that is not super hacky?

Thank you so much! 🙂

[gpleiss]

Hi @irinaespejo check out this section of the multitask SVGP tutorial: https://docs.gpytorch.ai/en/latest/examples/04_Variational_and_Approximate_GPs/SVGP_Multitask_GP_Regression.html#Output-modes

You'd do something like this:

task_indices = torch.cat([torch.zeros(n_1, dtype=torch.long), torch.ones(n_2, dtype=torch.long)])  # an index tensor that is [0, ..., 0, 1, ..., 1], that will assign each input to its respective task
model(torch.cat([x_1, x_2], dim=-2), task_indices=task_indices)  # outputs a multivariate normal of size n_1 + n_2