tfa.image.rotate Different Fill Modes #1318
Description
Describe the feature and the current behavior/state.
Currently, when an image is rotated using the tfa.image.rotate() function, the outer regions of the image are filled with 0's, resulting in an image with black corners usually. This is usually not preferred, especially when the user is trying to create new images via data augmentation. Images with black corners are usually not a good representation of the intended creation.
tf.keras.preprocessing.image.random_rotation has a really good option where you can control how exactly the blank space is filled. The following pictures shows some of the common methods and how the resulting image looks closer to the original.
It would be nice to see the same option added to tfa.image.rotate.
Relevant information
- Are you willing to contribute it (yes/no): Yes!
- Are you willing to maintain it going forward? (yes/no): Yes!
- Is there a relevant academic paper? (if so, where): No
- Is there already an implementation in another framework? (if so, where): Keras (obviously)
- Was it part of tf.contrib? (if so, where): Not that I am aware of
Which API type would this fall under (layer, metric, optimizer, etc.)
tfa.image
Who will benefit with this feature?
Anyone who is currently using tfa.image.rotate for data / image augmentation purposes and want different methods to fill the image boundaries to create better augmented images for training.
Any other info.
While tf.keras.preprocessing.image.random_rotation (assuming you are performing rotations randomly) is capable of doing the same thing, it is generally not convenient to use it in a data pipeline. For one reason, it's not really designed for tensors in the form of (num_pictures, num_rows, num_cols, num_channels), the channels last format which is whats normally expected. Secondly, it expects inputs as Numpy arrays. Finally it is hard to use directly with @tf.function. Thus, I think a native implementation would be preferred.