-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
I would like to contribute specialized positional encodings. This issue is meant to track the request and gather feedback from the community and maintainers before I open a pull request with the implementation.
Is your feature request related to a problem? Please describe.
Sinusoidal and Learned positional encodings (PEs) are often suboptimal for medical images. This is especially the case for anistropic images and for videos (see pre-print accepted in ShapeMI@MICCAI here).
Describe the solution you'd like
I would like to add the positional encodings, described in the pre-print:
- Isotropic Fourier Feature Positional Encodings (IFPE)
- Anistropic Fourier Feature Positional Encodings (AFPE)
They could be implemented in a single function and used in the PatchEmbeddingBlock
.
Describe alternatives you've considered
Alternatively the PEs would have to be imported from another library.