mode collapse / exploding gradients

[johndpope]

this happens now and then....

i dont know why - or when

https://wandb.ai/snoozie/IMF/runs/01vvfows?nw=nwusersnoozie

the lead up was showing convergence -

while the author says ADA wasn't used - it does have a feature in that it can prevent mode collapse.

AdaIN (Adaptive Instance Normalization) has a significant impact on StyleGAN2's capabilities and performance. Here are the key effects:

1. Improved style control:
   - AdaIN allows for better control over the styles at different scales/resolutions in the generator.
   - It enables more fine-grained manipulation of features like color, texture, and higher-level attributes.

2. Enhanced disentanglement:
   - AdaIN helps separate different aspects of the generated images, making it easier to independently control various attributes.

3. Better mixing capabilities:
   - Style mixing becomes more effective, allowing for more natural blending of features from different latent codes.

4. Increased stability:
   - AdaIN contributes to more stable training, reducing issues like mode collapse.

5. Improved quality:
   - Generally leads to higher quality and more diverse image generation.

6. Flexibility in architecture:
   - Allows for a more flexible generator architecture, as styles can be injected at multiple resolutions.

7. Enhanced transfer learning:
   - Makes it easier to adapt the model to new domains or tasks through transfer learning.

8. Better interpretability:
   - The style space becomes more interpretable, aiding in understanding what different parts of the latent space control.

9. Efficient scale-specific control:
   - Enables efficient control over features at different scales without needing to pass through the entire network.

10. Reduced artifact generation:
    - Can help reduce certain types of artifacts in generated images.

11. Improved editability:
    - Makes it easier to perform semantic edits on generated images by manipulating the style vectors.

12. Better performance on diverse datasets:
    - AdaIN helps the model handle more diverse datasets by allowing for more flexible style adjustments.

These improvements make StyleGAN2 with AdaIN a powerful and versatile architecture for high-quality image generation and manipulation tasks.

the root cause i think lies in the framedecoder

there's a step where the discriminator loss is off the charts....
is there something wrong with training data??

sometimes the videos have unrelated current / reference...

I did build out ada discriminator - maybe this could help....

I switch in the original IMF disicriminator
#35

seems to be helping
https://wandb.ai/snoozie/IMF/runs/nh3zc28s?nw=nwusersnoozie

this is using the multiscale discriminator
https://wandb.ai/snoozie/IMF/runs/191c5zqi?nw=nwusersnoozie

i though that this could make a big impact on image quality - but in fairness the correct architecture has been the most help.

[johndpope]

Fixed with learning rate clean up