How to use soft label to train or test

[MUVGuan]

@owl-10 Hello, thanks for your great work.
If I want to train MaskAdapter with another dataset where the annotations are not class indices but probabilities or heatmap, for example:
the shape of annotation: [10, 5, 5], 10 is number of classes, 5*5 is the size of image;
annotation[0] =
[0, 0, 150, 100, 0
0, 150, 255, 150, 0
0, 0, 150, 0, 0
0, 0, 0, 0, 0
0, 0, 0, 0, 0]
the range of pixel values in annotation[0] is [0, 255], this example represents the annotation of class 0 for the current image, other classes follow this setting.
Could you please give some guidances about how to change the code? Thank you very much.

[owl-10]

Based on your description, the annotation is a heatmap for each class. In your task, you can input the heatmap together with the CLIP features—into the Mask Adapter to extract semantic activation maps. These activation maps can then be aggregated with the CLIP features to obtain the object embedding. The resulting object embedding is subsequently matched with the text embedding to determine the category. Alternatively, you can set a fixed threshold to convert the heatmap annotation into a pseudo-mask. Then you can training as default.