Numpy de/serialization is buggy for XGBoost endpoint

[tigerhawkvok]

Reference: 0420648124

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): XGBoost
• Framework Version: 0.90
• Python Version: 3
• CPU or GPU: CPU
• Python SDK Version: Latest
• Are you using a custom image:

Describe the problem

Using the baked-in de/serializers don't work:

from sagemaker.predictor import RealTimePredictor
from sagemaker.predictor import numpy_deserializer, npy_serializer
realPredictor = RealTimePredictor(endpointName, serializer= npy_serializer, deserializer= numpy_deserializer)

has predictions throw ModelError: [...] "content-type application/x-npy not supported"

"rolling my own" pulling from examples just has a generic payload failure:

def _npy_dumps(data):
    """
    Serializes a numpy array into a stream of npy-formatted bytes.
    """
    from six import BytesIO
    import numpy as np
    buffer = BytesIO()
    np.save(buffer, data)
    return buffer.getvalue()

def _npy_loads(data):
    """
    Deserializes npy-formatted bytes into a numpy array
    """
    from six import BytesIO
    import numpy as np
    stream = BytesIO(data)
    return np.load(stream)
                   
realPredictor = RealTimePredictor(endpointName, serializer= _npy_dumps, deserializer= _npy_loads)
realPredictor.predict(testRow)
# ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message "unable to evaluate payload provided"

which CloudWatch elaborates as UnicodeDecodeError: 'utf8' codec can't decode byte 0x93 in position 0: invalid start byte

For what it's worth, an explicitly-casted list of floats also fails, but it kind of seems like it should without a similar serializer

testRow = [float(x) for x in testRow.tolist()]
realPredictor = RealTimePredictor(endpointName)
realPredictor.predict(testRow)

"""
ParamValidationError: Parameter validation failed:
Invalid type for parameter Body, value: [-0.06891748633639135, -0.066079594710651, -0.0025436381630378753, -0.03100791692543576, 0.18405785293524227, -0.5850985235117403, -0.11973680865060594, -0.5311303586807635, -0.6137469447090678, -0.34089114681717464, -0.17146581887585718, -0.35787901414757445, 1.0640251382364134, -1.0640251382364134, 0.8621821650983366, -0.3989759381383541, -0.9338172571867578, 0.29422864222294615, -0.4430163494426008, -0.041651324851978914, 0.6226488370425939, 1.2676271211973693, 0.9856193979982015, -0.2211529817662006, -0.2029367205962168, 0.8399726034068296, -0.8399726034068297, -1.9561990128264775, 1.9561990128264775, -1.2671645025215617, 1.2671645025215619, 0.47424758792111377, -0.47424758792111377], type: <class 'list'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object
"""

Following up on that error, trying a file-like object also fails

dumpable = pd.DataFrame(testRow)
dumpable.to_csv("test.csv", header= False, index= False)
with open("test.csv", encoding='utf-8') as fh:
    realPredictor.predict(fh)
    # TypeError: Unicode-objects must be encoded before hashing

This estimator was set up directly via the SageMaker GUI after failing to programmatically set up an endpoint as per #1135

[ChoiByungWook]

Reference: 0420648124

[TomLaMantia]

I had a similar issue with the Sagemaker SKLearn estimator.

After deploying the model as follows:

predictor = clf.deploy(instance_type='local_cpu', initial_instance_count=1, endpoint_name=endpoint_name)

and predicting as follows:

predictor.predict(my_np_array)

I would get the following error: ValueError: Object arrays cannot be loaded when allow_pickle=False. This was occurring when the default serializer attempted to load a stream of bytes to a numpy array. While the docs claim that, "The SageMaker Scikit-learn model server provides a default implementation of input_fn. This function deserializes JSON, CSV, or NPY encoded data into a NumPy array.", there seems to be issues if the input is an array of strings.

I fixed the issue by writing my own deseralizer similar to what @tigerhawkvok suggests, but setting this as an attribute of the RealTimePredictor object (perhaps you can do something similar with your endpoint):

def custom_np_deserializer(np_bytes, content_type='application/x-npy'):
    # TO-DO: Test content_type and raise appropriate exception when required
    return np.load(io.BytesIO(np_bytes.read()), allow_pickle=True)

predictor.deserializer = custom_np_deserializer

and predicting as follows:

predictor.predict(my_np_array)

I was stuck on this for quite a while, so I hope that helps?

[benieric]

Hey @tigerhawkvok, the RealTimePredictor and numpy_deserializer classes have been deprecated/renamed to Predictor and NumpyDeserializer() in versions 2.x of the SageMaker Python SDK. Do you know if this issue is still relevant with newer versions of the sdk?

[zhaoqizqwang]

Resolving this issue since it's deprecated. Feel free to reopen if it happens in new classes.