Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Add T5 Model to TorchText #1845

Merged
merged 17 commits into from
Jul 18, 2022
Merged

Add T5 Model to TorchText #1845

merged 17 commits into from
Jul 18, 2022

Conversation

@pmabbo13
Copy link
Contributor

@pmabbo13 pmabbo13 commented Jul 18, 2022 •
edited
Loading

Original stack of PRs can be found here: #1825
This PR encapsulates all of the changes made in those PRs in order to merge them into the main branch.

pmabbo13 and others added 17 commits July 12, 2022 17:35
@pmabbo13
compute relative position buckets for relative attention bias
70dad25 
[ghstack-poisoned]
@pmabbo13
compute relative position bias for t5 attention
88743d6 
[ghstack-poisoned]
@pmabbo13
compute attention scores for t5 model using relative attention bias
7b67d56 
[ghstack-poisoned]
@pmabbo13
perform multihead attention using relative attention bias for t5 model
f3fac0e 
[ghstack-poisoned]
@pmabbo13
create T5MultiheadAttention module
13f9c22 
[ghstack-poisoned]
@pmabbo13
add layer norm module for t5 model
1f99650 
[ghstack-poisoned]
@pmabbo13
add t5 layer module that can be used for both encoder or decoder stack
12e48e1 
[ghstack-poisoned]
@pmabbo13
add t5 stack that can function as either the encoder or decoder of a …
be54aef 
…t5 model

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
dc55ec9 
…odery-only or encoder-decoder model"

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
a213d8c 
…odery-only or encoder-decoder model"

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
c48d292 
…odery-only or encoder-decoder model"

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
4e661a0 
…odery-only or encoder-decoder model"

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
ff1f0e8 
…odery-only or encoder-decoder model"

[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
6b77460 
…odery-only or encoder-decoder model"




[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
eef9916 
…odery-only or encoder-decoder model"




[ghstack-poisoned]
@pmabbo13
Update base for Update on "add t5 model that can function as both enc…
88bf197 
…odery-only or encoder-decoder model"




[ghstack-poisoned]
@pmabbo13
add t5 model that can function as both encodery-only or encoder-decod…
adbc511 
…er model (#1829)
Nayef211
Nayef211 approved these changes Jul 18, 2022
View reviewed changes
Copy link
Contributor

@Nayef211 Nayef211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pmabbo13 pmabbo13 merged commit 8d56ed2 into main Jul 18, 2022
@facebook-github-bot facebook-github-bot deleted the gh/pmabbo13/9/base branch August 18, 2022 14:20
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@Nayef211 Nayef211 Nayef211 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla signed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pmabbo13 @Nayef211 @facebook-github-bot