When Fast Fourier Transform Meets Transformer for Image Restoration (2024)

5 comments

• TimorousBestie4 hours ago
  There have been some interesting advances in trying to add spectral information to the data that a learning architecture has at its disposal, but there are a couple roadblocks that I don’t think have been solved yet.<p>1. Complex-valued NNs are not an easy generalization of real ones.<p>2. A localization in one domain implies non-local behavior in the other (this is the Fourier uncertainty principle).<p>Fourier Neural Operators (FNOs) come close to what I want to see in this area but since they enforce sparsity in the spectral domain their application is necessarily limited.
  • FuckButtons3 hours ago
    I do wonder if a wavelet transform might be better.
    • TimorousBestie1 hour ago
      I think one can do better with a wavelet, shearlet, or curvelet transform that is adapted to the problem domain at hand. But the uncertainty principle still haunts those transforms, and anyway the goal is to be domain-agile.
• waynecochran1 hour ago
  Was there a conclusion?
• sorenjan3 hours ago
  See also: CosAE: Learnable Fourier Series for Image Restoration (2024)<p><a href="https:&#x2F;&#x2F;sifeiliu.net&#x2F;CosAE-page&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sifeiliu.net&#x2F;CosAE-page&#x2F;</a>
• gryfft5 hours ago
  [2024]