[ECCV 2024] Official PyTorch implementation of RoPE-ViT "Rotary Position Embedding for Vision Transformer"
-
Updated
Oct 29, 2025 - Python
#
rotary-position-embedding
Here are 5 public repositories matching this topic...
[ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
-
Updated
Mar 12, 2024 - Python
PyTorch implementation of Rotary Spatial Embeddings
pytorch transformer attention positional-encoding rotary-position-embedding rotary-position-encoding
-
Updated
Sep 27, 2025 - Python
Decoder-only LLM trained on the Harry Potter books.
transformer byte-pair-encoding rotary-position-embedding large-language-model flash-attention grouped-query-attention
-
Updated
Dec 20, 2024 - Python
A from-scratch implementation of a T5 model modified with Rotary Position Embeddings (RoPE). This project includes the code for pre-training on the C4 dataset in streaming mode with Flash Attention 2.
nlp pytorch sequence-to-sequence language-model from-scratch rope pre-training huggingface t5 evaluation-benchmark llm rotary-position-embedding flash-attention c4-dataset span-corruption
-
Updated
Jul 9, 2025 - Python
Improve this page
Add a description, image, and links to the rotary-position-embedding topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the rotary-position-embedding topic, visit your repo's landing page and select "manage topics."