Skip to content

Commit b874f0b

Browse files
authored
Update noisy_dense.py
1 parent 4fb4bff commit b874f0b

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

tensorflow_addons/layers/noisy_dense.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,15 @@
2828
class NoisyDense(tf.keras.layers.Layer):
2929
"""Like normal dense layer (https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/python/keras/layers/core.py#L1067-L1233)
3030
but random noisy is added to the weights matrix. But as the network improves the random noise is decayed until it is insignificant.
31+
3132
A `NoisyDense` layer implements the operation:
3233
`output = activation(dot(input, µ_kernel + (σ_kernel * ε_kernel)) + bias)`
3334
where `activation` is the element-wise activation function
3435
passed as the `activation` argument, `µ_kernel` is your average weights matrix
3536
created by the layer, σ_kernel is a weights matrix that controls the importance of
3637
the ε_kernel which is just random noise, and `bias` is a bias vector created by the layer
3738
(only applicable if `use_bias` is `True`).
39+
3840
Example:
3941
>>> # Create a `Sequential` model and add a Dense layer as the first layer.
4042
>>> model = tf.keras.models.Sequential()
@@ -47,6 +49,7 @@ class NoisyDense(tf.keras.layers.Layer):
4749
>>> model.add(NoisyDense(32))
4850
>>> model.output_shape
4951
(None, 32)
52+
5053
Arguments:
5154
units: Positive integer, dimensionality of the output space.
5255
activation: Activation function to use.
@@ -61,10 +64,12 @@ class NoisyDense(tf.keras.layers.Layer):
6164
kernel_constraint: Constraint function applied to
6265
the `kernel` weights matrix.
6366
bias_constraint: Constraint function applied to the bias vector.
67+
6468
Input shape:
6569
N-D tensor with shape: `(batch_size, ..., input_dim)`.
6670
The most common situation would be
6771
a 2D input with shape `(batch_size, input_dim)`.
72+
6873
Output shape:
6974
N-D tensor with shape: `(batch_size, ..., units)`.
7075
For instance, for a 2D input with shape `(batch_size, input_dim)`,

0 commit comments

Comments
 (0)