I don't understand why you upsample to the shape 304*228 instead of upsampling to the original image size of shape (640, 480) to compute the loss.