Error in MultiOptimizer when layers list are used

[guillaumelorre28]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
• TensorFlow version and how it was installed (source or binary): docker tensorflow
• TensorFlow-Addons version and how it was installed (source or binary): binary
• Is GPU used? (yes/no):

Describe the bug

Documentation of MultiOptimizer explains that list of layers can be used but it results in an error.

addons/tensorflow_addons/optimizers/discriminative_layer_training.py

Line 38 in c7b867a

layers and models will be automatically discovered. Alternatively, in place of a single layer, you can pass

addons/tensorflow_addons/optimizers/discriminative_layer_training.py

Line 52 in c7b867a

opt_layer_pairs = [(opt1, model.layers[0]), (opt2, model.layers[1:])]

Code to reproduce the issue

import tensorflow as tf
import tensorflow_addons as tfa
import numpy as np


model = tf.keras.Sequential(
        [tf.keras.Input(shape=[1]), tf.keras.layers.Dense(1), tf.keras.layers.Dense(1), tf.keras.layers.Dense(1)]
    )

x = np.array(np.ones([100]))
y = np.array(np.ones([100]))

weights_before_train = (
    model.layers[0].weights[0].numpy(),
    model.layers[1].weights[0].numpy(),
)

opt1 = tf.keras.optimizers.Adam(learning_rate=1e-3)
opt2 = tf.keras.optimizers.SGD(learning_rate=0)

opt_layer_pairs = [(opt1, model.layers[0]), (opt2, model.layers[1:])]

loss = tf.keras.losses.MSE
optimizer = tfa.optimizers.MultiOptimizer(opt_layer_pairs)

model.compile(optimizer=optimizer, loss=loss)

Error produced

Traceback (most recent call last):
  File "test_multi_optimizer.py", line 24, in <module>
    optimizer = tfa.optimizers.MultiOptimizer(opt_layer_pairs)
  File "/usr/local/lib/python3.6/dist-packages/typeguard/__init__.py", line 840, in wrapper
    retval = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_addons/optimizers/discriminative_layer_training.py", line 92, in __init__
    for opt, layer in optimizers_and_layers
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_addons/optimizers/discriminative_layer_training.py", line 92, in <listcomp>
    for opt, layer in optimizers_and_layers
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_addons/optimizers/discriminative_layer_training.py", line 142, in create_optimizer_spec
    ), "Object passed is not an instance of tf.keras.layers.Layer nor tf.keras.Model"
AssertionError: Object passed is not an instance of tf.keras.layers.Layer nor tf.keras.Model

[bhack]

/cc @hyang0129
Yes, also I see only the layers list logic but not the models list.

[guillaumelorre28]

I removed this line in the code and it works fine.

addons/tensorflow_addons/optimizers/discriminative_layer_training.py

Line 140 in c7b867a

assert isinstance(layer, tf.keras.layers.Layer) or isinstance(

assert isinstance(layer, tf.keras.layers.Layer) or isinstance(
            layer, tf.keras.Model
        ), "Object passed is not an instance of tf.keras.layers.Layer nor tf.keras.Model"

[bhack]

The assert part is easy because you need to just add an "or" check that isinstance of list and that every element is an isinstance of layer or Model,
But what I see is that is missing the list of models code after the assert.

[bhack]

I don't know what the original idea was but I've tried to put some fixes at #2179