SAM finetuning issue

[mincheoree]

Thanks for great code sharing. @JunMa11
I got a question about SAM finetuning on custom dataset other than medical data.
My dataset consists of 300+images, and each image contains 100+ small instances to segment.
So each of my gt2D masks consist of small number of 1s (foreground) and large number of 0s (background).
Following your codes, I designed finetuning framework for my dataset.
I used dice and cross entropy losses to train the mask decoder of SAM.
However, it seems like my model learns to segment less and less as the epochs increase.
In the end, finetuned model gets worse than baseline SAM.
Is this because the nature of my dataset different from medical data, in which the target instances are relatively large.
It seems like my loss is small, and model learns to not segment. I have visualized and checked the gt2Ds and predictions are correct.
Do you have any suggestions for this case?

[JackWei01]

Thanks for great code sharing. @JunMa11 I got a question about SAM finetuning on custom dataset other than medical data. My dataset consists of 300+images, and each image contains 100+ small instances to segment. So each of my gt2D masks consist of small number of 1s (foreground) and large number of 0s (background). Following your codes, I designed finetuning framework for my dataset. I used dice and cross entropy losses to train the mask decoder of SAM. However, it seems like my model learns to segment less and less as the epochs increase. In the end, finetuned model gets worse than baseline SAM. Is this because the nature of my dataset different from medical data, in which the target instances are relatively large. It seems like my loss is small, and model learns to not segment. I have visualized and checked the gt2Ds and predictions are correct. Do you have any suggestions for this case?

Hello, I also have the same need recently. However, the files of the dataset in the warehouse (GoogleDriver) seem to be invalid. May I ask if you have the file format of the dataset for fine - tuning training?