Implement LazyAdamOptimizer #11
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facaiy
force-pushed
the
ENH/move_LazyAdamOptimier
branch
from
c4a3b9f to
1922bbc
Compare
Member
Author
|
@seanpmorgan @armando-fandango Could you take a look? Thanks :) |
facaiy
commented
Jan 14, 2019
tensorflow_addons/optimizers/python/lazy_adam_optimizer_test.py
Outdated
Show resolved
Hide resolved
seanpmorgan
requested changes
Jan 14, 2019
Member
seanpmorgan
left a comment
seanpmorgan
left a comment
There was a problem hiding this comment.
Thanks very much for the PR @facaiy. Generally looks good, few stylistic changes requested.
tensorflow_addons/optimizers/python/lazy_adam_optimizer_test.py
Outdated
Show resolved
Hide resolved
facaiy
force-pushed
the
ENH/move_LazyAdamOptimier
branch
from
370b575 to
cfba7d3
Compare
Member
Author
|
Thank you, Sean. For v1 tests, I'd like to keep the same pace with keras.optimizer to remove them. I think I have addressed all your comments, could you take a look again? |
facaiy
force-pushed
the
ENH/move_LazyAdamOptimier
branch
3 times, most recently
from
8c3821b to
06f8176
Compare
facaiy
force-pushed
the
ENH/move_LazyAdamOptimier
branch
from
06f8176 to
76843b2
Compare
seanpmorgan
approved these changes
Jan 15, 2019
Squadrick
pushed a commit
to Squadrick/addons
that referenced
this pull request
Mar 26, 2019
Implement LazyAdamOptimizer
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #4
The implementation are based on tensorflow/contrib/opt/python/training/lazy_adam_optimizer.py and tensorflow/python/keras/optimizer_v2/adam.py
The test codes are based on tensorflow/contrib/opt/python/training/lazy_adam_optimizer_test.py and tensorflow/python/keras/optimizer_v2/adamax_test.py
Note that RefVariable is not supported any more. They are modified to ensure that all tests pass in nightly-custom-op image.