-
Notifications
You must be signed in to change notification settings - Fork 617
Closed
Description
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
- TensorFlow version and how it was installed (source or binary): source, nightly
- TensorFlow-Addons version and how it was installed (source or binary): source, nightly
- Python version: 3.6.7
- Is GPU used? (yes/no): yes
Describe the bug
When restoring the RectifiedAdam optimizer from a checkpoint, restoring fails with the following error message:
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype int32: <tf.Tensor: shape=(), dtype=int32, numpy=0>
The reason is that RectifiedAdam stores total_steps as an integer hyper-parameter:
https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/rectified_adam.py#L134
OptimizerV2 however creates variables by calling add_weight without a dtype in _create_hypers. In other words, it assumes that all hyper-parameters are floating point parameters.
Storing the total_steps parameter as float fixes the problem for me.
Metadata
Metadata
Assignees
Labels
No labels