Endpoint returning a "input tensor alias not found in signature" error

[jonsnowseven]

Please fill out the form below.

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): TensorFlow
• Framework Version: 1.6.0 (not sure)
• Python Version: 3.6
• CPU or GPU: ...
• Python SDK Version: 1.2.3
• Are you using a custom image: No

Describe the problem

I am trying to use SageMaker end-to-end.

Training:

from sagemaker.tensorflow import TensorFlow

job_name = ****

estimator = TensorFlow(
    entry_point='sagemaker-script.py',
    source_dir=****,
    role=role,
    training_steps=1000,
    evaluation_steps=100,
    hyperparameters={
        'learning_rate': 1e-04,
        'input_layer': 'inputs',
        'input_layer_full_name': 'inputs_input',
        'max_len': 42
    },
    train_instance_count=1,
    train_instance_type='ml.p3.2xlarge',
    checkpoint_path=****)

estimator.fit(****, job_name=job_name)

predictor = estimator.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge')

predict_data = ****

predictor.predict(predict_data.X)

Where sagemaker-script.py is:

from __future__ import print_function, unicode_literals

import os
import pandas as pd
import numpy as np
import tensorflow as tf
import yaml

...

def create_corpus(path):
    ****
    return text


def keras_model_fn(hyperparameters):
    log.info('Calling keras_model_fn')

    ****
    return model


def train_input_fn(training_dir=None, hyperparameters=None):
    X, y = _input_fn(training_dir, hyperparameters)
    return tf.estimator.inputs.numpy_input_fn(
        x={hyperparameters['input_layer_full_name']: X},
        y=y,
        num_epochs=None,
        shuffle=True)()


def _input_fn(training_dir, hyperparameters):
    ****
    train_data_gen = ****
    return train_data_gen.X, train_data_gen.y


def eval_input_fn(training_dir=None, hyperparameters=None):
    log.info("Calling eval_input_fn")

    X, y = _eval_fn(training_dir, hyperparameters)
    log.info("SIGNATURE: {}".format(hyperparameters['input_layer_full_name']))

    log.info("eval_input_fn DONE")
    return tf.estimator.inputs.numpy_input_fn(
        x={hyperparameters['input_layer_full_name']: X},
        y=y,
        num_epochs=None,
        shuffle=True)()


def _eval_fn(training_dir, hyperparameters):
    ****
    val_data_gen = ****
    return val_data_gen.X, val_data_gen.y


def serving_input_fn(hyperparameters):

    char_indices = ****

    # defines the input placeholder
    tensor = tf.placeholder(tf.int8, shape=[None, hyperparameters['max_len'], len(char_indices)])

    serving_input_receiver = tf.estimator.export.build_raw_serving_input_receiver_fn(
        {hyperparameters['input_layer_full_name']: tensor})()

    # returns the ServingInputReceiver object.
    return serving_input_receiver

Minimal repo / logs

The prediction command results in the following:

AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input tensor alias not found in signature: inputs. Inputs expected to be in the set {inputs_input}.")
[2018-04-26 13:42:32,944] ERROR in serving: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input tensor alias not found in signature: inputs. Inputs expected to be in the set {inputs_input}.")

Can you help me?

Thank you.

[lukmis]

Hi jonsnowseven,

The default TF container supports python2 so please make sure your code is compatible.

The error message suggests the issue is with 'serving_input_fn' code. The examples I saw were using 'tf.estimator.export.ServingInputReceiver'. Have you tried to use that one instead?

In general, you'd do something like this inside your 'serving_input_fn':

tensor = tf.placeholder(tf.int8, shape=[None, hyperparameters['max_len'], len(char_indices)])
inputs = {INPUT_TENSOR_NAME:tensor}
return tf.estimator.export.ServingInputReceiver(inputs, inputs)

Please let us know if this is helpful.

[jonsnowseven]

Hi @lukmis.

Thank you for your answer.

I saw a lot of examples using the function tf.estimator.export.build_raw_serving_input_receiver_fn, which returns a function such that when called returns a tf.estimator.export.ServingInputReceiver. But I can try like that. I've been stuck on this error for some days, so everything new is helpful.

I'll try it like that and report the results.

Thank you once again.

[jonsnowseven]

Hi @lukmis.

What you proposed did not work.

The weird thing is that I am respecting the protocol, meaning that I am building the dict {keyword: tensor} with the correct keyword. But when I perform the request for prediction, "something" is sending a wrong keyword inputs instead of the one expected: inputs_input.

[lukmis]

Hi @jonsnowseven,

I only now realized that your the issue happens during prediction not training. So when you say that you call:
predictor.predict(predict_data.X)
it's important to understand what is that you are passing. I assume this conforms to what the model is expecting (and probably matches train_data_gen.X).
Additionally, Keras takes the name of the first layer and adds '_input' and that is why the message complains about 'inputs_input'. That makes me think this is a lead on your issue here.
So during prediction, the data you sent is by default serialized into protobuf in json and then in the container deserialized back. The tensor has the 'default' name of inputs and it doesn't match what Keras is expecting.
So in your case you might need to write your own input_fn() that will massage the data into the expected format. Please refer to the README for more details how this can be done.

Please let us know if this direction works for you.

[jonsnowseven]

Hello @lukmis.

Yes, I am sorry if that was not clear. It is during prediction this issues happens.

I previously thought of defining my own input_fn (and output_fn eventually) but I took another direction which was changing from keras_model_fn to model_fn using a Keras model. I am not aware of the differences in doing this. But after doing this prediction worked.

def model_fn(features, labels, mode, params):
    """Model function for Estimator."""

    # Configure the model via Keras functional api
    output_layer = ****

    predictions = tf.reshape(output_layer, [-1])

    # Provide an estimator spec for `ModeKeys.PREDICT`.
    if mode == tf.estimator.ModeKeys.PREDICT:
        predictions_dict = ****
        return tf.estimator.EstimatorSpec(
            mode=mode,
            predictions=predictions_dict,
            export_outputs=****)

    # 2. Define the loss function for training/evaluation using Tensorflow.
    loss = ****

    # Define the training operation/optimizer using Tensorflow operation/optimizer.
    train_op = ****

    # Generate necessary evaluation metrics.
    eval_metric_ops = ****

    # Provide an estimator spec
    return tf.estimator.EstimatorSpec(
        mode=mode,
        loss=loss,
        train_op=train_op,
        eval_metric_ops=eval_metric_ops)

If I have some time, I'll try defining input_fn at least at test this.

[Brunods1001]

Hello @jonsnowseven, have you had any luck getting your keras_model_fn to work? I am facing the same problem with my keras_model_fn:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import print_function

import numpy as np
import os
import pandas as pd
import tensorflow as tf
from tensorflow.python.estimator.export.export import build_raw_serving_input_receiver_fn
from tensorflow.python.estimator.export.export_output import PredictOutput

# Embedding
MAX_FEATURES = 20000
MAX_LENGTH = 100
EMBEDDING_SIZE = 200

# Convolution
KERNEL_SIZE = 5
FILTERS = 64
POOL_SIZE = 4

INPUT_TENSOR_NAME = 'inputs_input' # needs to match the name of the first layer + "_input"
SIGNATURE_NAME = "serving_default"
LEARNING_RATE = 0.001
CLASSES = 5


def keras_model_fn(params):    
    model = tf.keras.models.Sequential()
    model.add(tf.keras.layers.Embedding(MAX_FEATURES, EMBEDDING_SIZE, input_length=MAX_LENGTH, name='inputs'))
    model.add(tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(300, return_sequences=True, dropout=0.1, recurrent_dropout=0.1)))
    model.add(tf.keras.layers.GlobalMaxPool1D())
    model.add(tf.keras.layers.Dense(50, activation="relu"))
    model.add(tf.keras.layers.Dropout(0.1))
    model.add(tf.keras.layers.Dense(5, activation="softmax"))
    optim = tf.keras.optimizers.Adam(lr=0.0001)
    model.compile(optimizer=optim, loss="categorical_crossentropy", 
                  metrics=["accuracy"])
    return model

def serving_input_fn(params):
    tensor = tf.placeholder(tf.int32, shape=[1, MAX_LENGTH])
    return build_raw_serving_input_receiver_fn({INPUT_TENSOR_NAME: tensor})()
   
def train_input_fn(training_dir, hyperpameters):
    return _input_fn(training_dir, 'train_source.csv')

def eval_input_fn(training_dir, hyperpameters):
    return _input_fn(training_dir, 'train_source.csv')

def _input_fn(training_dir, training_filename):
    '''
    Helper function used to clean text for training and eval inputs
    '''
    training_set = tf.contrib.learn.datasets.base.load_csv_without_header(
        filename=os.path.join(training_dir, training_filename), target_dtype=np.int, features_dtype=str)

    texts = training_set.data
    texts = pd.Series(texts.flatten())
    labels = training_set.target
    labels = labels.reshape([-1, 1])
    tokenizer = tf.keras.preprocessing.text.Tokenizer(num_words=MAX_FEATURES)
    tokenizer.fit_on_texts(texts)
    sequences = tokenizer.texts_to_sequences(texts)

    data = tf.keras.preprocessing.sequence.pad_sequences(sequences, maxlen=MAX_LENGTH)
    X = np.array(data, dtype=np.int32)
    Y = np.array(labels).astype(int)
    CLASSES = len(np.unique(Y))

    return tf.estimator.inputs.numpy_input_fn(
        x={INPUT_TENSOR_NAME: X},
        y=Y,
        num_epochs=None,
        shuffle=True)()

[jonsnowseven]

Hello @Brunods1001.

Sorry for not sharing news but I have be loaded up in work.

I made it work replacing keras_model_fn for model_fn (you can see my last message).

Although, I think defining the input_fn plus keras_model_fn functions might be my next try, when I have the time.

I'll keep you posted.

Best regards,

João Neves

[nadiaya]

Hello @jonsnowseven,

Are you still experiencing the issue? Please, let us know if you still require our assistance.

[jonsnowseven]

Hello @nadiaya.

I am experiencing the same issue, as stated originally.

As I said in my previous message, changing keras_model_fn to model_fn solved my problem and I was able to predict through the endpoint.

Although, I do not now the difference between keras_model_fn and model_fn.

Do you know?

Thank you.

[whachyzachy]

I am experiencing the same issue.

[andremoeller]

Hi @whachyzachy , @jonsnowseven , and @Brunods1001 ,

That INPUT_TENSOR_NAME should be equal to the name of your first layer + "_input" only applies if your first layer is a tf.keras.layers.InputLayer.

[jonsnowseven]

Hello @andremoeller.

Didn't you mean "name of your first layer + _input" (in singular form)?

If that is the case that is what I am doing:

...
    hyperparameters={
        'learning_rate': 1e-04,
        'input_layer': 'inputs',
        'input_layer_full_name': 'inputs_input', # this name is being used everywhere.
        'max_len': 42
    },
...

If you mean "name of your first layer + _inputs" then the resulting Keras layer name would be inputs_inputs (at least in my case).