WaveFT Integration

[Bilican]

Feature request

The following paper with the corresponding code propose a new PEFT method which demonstrates superior performance to existing methods (Adalora, VeRA, LoRA, LoHA, LoKR, FourierFT) in vision tasks. In the repository one can easily reproduce every result in the given paper because all hyperparameters and training scripts for each method are shared. The training and evaluation scripts already use PEFT implementations for each method.

Motivation

Both of the proposed methods, WaveFT and SHiRA have advantages compared to other methods most significantly in generating diverse outputs while preserving subject fidelity. For computational efficiency the wavelet transform part can be disabled (SHiRA) which results in a loss of performance but is still better than other adaptors. Both of these methods have use cases in different scenarios.

Your contribution

I have already written the code in the repository to be compatible and follow the PEFT conventions. It would be great if you could directly review the implemented files, but if necessary I could submit a PR to help. The method is implemented here.