sagemaker.workflow.conditions.ConditionNot does not seem to be working

[DavidSAmundsen]

Describe the bug

I would like to use sagemaker.workflow.conditions.ConditionNot to negate a sagemaker.workflow.conditions.ConditionEquals. However, use of sagemaker.workflow.conditions.ConditionNot triggers the following exception:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation: Unable to parse pipeline definition. Cannot parse object, unknown property 'Expression'.

The workaround we are using for now is to use else_steps instead of if_steps in the ConditionStep, and avoid using ConditionNot.

To reproduce

The following Python script can be used to reproduce the bug:

import sagemaker
from sagemaker.workflow.condition_step import ConditionStep
from sagemaker.workflow.conditions import ConditionEquals, ConditionNot
from sagemaker.workflow.parameters import ParameterBoolean
from sagemaker.workflow.pipeline import Pipeline


def get_pipeline() -> Pipeline:
    param_bool = ParameterBoolean("BoolParam", default_value=True)

    cond = ConditionNot(ConditionEquals(left=param_bool, right=True))
    step_cond = ConditionStep(
        name="ConditionStep",
        conditions=[cond],
    )

    return Pipeline(
        name="test-pipeline",
        parameters=[param_bool],
        steps=[step_cond]
    )


if __name__ == "__main__":
    role = sagemaker.get_execution_role()

    pipeline = get_pipeline()
    response = pipeline.upsert(role_arn=sagemaker.get_execution_role())

Expected behavior

I would expect the above Python script to upsert the pipeline without any exceptions.

Screenshots or logs

Full traceback:

Traceback (most recent call last):
  File "test_pipeline.py", line 28, in <module>
    response = pipeline.upsert(role_arn=sagemaker.get_execution_role())
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/site-packages/sagemaker/workflow/pipeline.py", line 180, in upsert
    response = self.create(role_arn, description, tags)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/site-packages/sagemaker/workflow/pipeline.py", line 116, in create
    return self.sagemaker_session.sagemaker_client.create_pipeline(**kwargs)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/site-packages/botocore/client.py", line 388, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/site-packages/botocore/client.py", line 708, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation: Unable to parse pipeline definition. Cannot parse object, unknown property 'Expression'.

System information

A description of your system. Please provide:

• SageMaker Python SDK version 2.67.0
• Python version 3.8.5

Thanks in advance for taking the time to look at this report.

[akavipat]

Have the same problem. Sagemaker v 2.74

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation: Unable to parse pipeline definition. Cannot parse object, unknown property 'Expression'.

"Arguments": {
                "Conditions": [
                    {
                        "Type": "Not",
                        "Expression": {
                            "Type": "Or",
                            "Conditions": [
                                {
                                    "Type": "Equals",
                                    "LeftValue": {
                                        "Get": "Parameters.Stage1Run"
                                    },
                                    "RightValue": true
                                },
                                {
                                    "Type": "Equals",
                                    "LeftValue": {
                                        "Get": "Parameters.Stage2Run"
                                    },
                                    "RightValue": true
                                },
                                {
                                    "Type": "Equals",
                                    "LeftValue": {
                                        "Get": "Parameters.Stage3Run"
                                    },
                                    "RightValue": true
                                }
                            ]
                        }
                    }
                ],

[FlorentF9]

This bug still exists in sagemaker v2.76

[qidewenwhen]

Adding a note:
This is a bug in SDK:

• In SDK, ConditionNot has an attribute called expression, which is serialized to Expression in the Json
  

  sagemaker-python-sdk/src/sagemaker/workflow/conditions.py

  Lines 256 to 260 in d355d5b

  	self.expression = expression
  	def to_request(self) -> RequestType:
  	"""Get the request structure for workflow service calls."""
  	return {"Type": self.condition_type.value, "Expression": self.expression.to_request()}

• However, in backend, it's expecting a field called Condition rather than Expression

Tracking the fix in backlog