Tensorflow model can not be serve on Tensorflow Serving on Sagemaker SDK 1.5.1

[mondor]

Please fill out the form below.

System Information

• Tensorflow / Algorithm LSTM
• Tensorflow 1.8
• Python 2.7.15
• GPU instance:
• Python SDK Version: 1.5.1:
• Not using a custom image:

Describe the problem

The Tensorflow Model we build via Sagemaker can not be serve on Sagemaker's Tensorflow Serving, we have downgrade the Python SDK version to 1.4.2, which fixed the issue.

Minimal repro / logs

2018-06-26 06:37:58.550444: E tensorflow_serving/util/retrier.cc:38] Loading servable: {name: generic_model version: 1529994753} failed: Not found: Op type not registered 'ClipByValue' in binary running on KDVACOwB.aws.local. Make sure the Op and Kernel are registered in the binary running in this process.

[nadiaya]

Hi @mondor,

SDK versions 1.4.2 and 1.5.1 by default use different containers with different tensorflow versions: 1.4.2 uses TF-1.6 and 1.5.1 uses TF-1.8.

When have you trained the model and on what SDK version?
How are you deploying this model? For example, are you training and deploying it right away, attaching to an existing training job or creating a model object?

User script code and more detailed stacktrace would be really useful for debugging.

[laurenyu]

Closing due to inactivity. Feel free to reopen if necessary.