Skip to content

Failing to restore RectifiedAdam optimizer from checkpoint #2361

@olesalscheider

Description

@olesalscheider

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
  • TensorFlow version and how it was installed (source or binary): source, nightly
  • TensorFlow-Addons version and how it was installed (source or binary): source, nightly
  • Python version: 3.6.7
  • Is GPU used? (yes/no): yes

Describe the bug

When restoring the RectifiedAdam optimizer from a checkpoint, restoring fails with the following error message:
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype int32: <tf.Tensor: shape=(), dtype=int32, numpy=0>

The reason is that RectifiedAdam stores total_steps as an integer hyper-parameter:
https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/rectified_adam.py#L134

OptimizerV2 however creates variables by calling add_weight without a dtype in _create_hypers. In other words, it assumes that all hyper-parameters are floating point parameters.

Storing the total_steps parameter as float fixes the problem for me.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions