path is removed from entrypoint when using airflow operator #770
Description
Please fill out the form below.
System Information
- Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans):TensorFlow, Airflow
- Framework Version: Airflow 1.10.1, tensorflow 1.13.1
- Python Version: 3.7.1
- CPU or GPU: CPU
- Python SDK Version:1.18.11
- Are you using a custom image:No
Describe the problem
I want to use the Airflow operator 'SageMakerTrainingOperator'. When i define a path in the entypoint e.g. /my/path/to/entrypoint.py it will be converted to entrypoint.py.
This happens in the sagemaker/workkflow/airflow.py file in the function 'prepare_framework' (line 37)
script = os.path.basename(estimator.entry_point)
According to the documentary it should be possible to define entrypoints with relative or absolute paths.
Minimal repro / logs
entry_path='/my/path/train.py'
tf_estimator = TensorFlow(
entry_point=entry_path,
role=role,
train_instance_type='ml.m5.large',
train_instance_count=1,
framework_version='1.13.1',
py_version='py3',
model_dir='s3://some_dir'
)
inputs = {}
train_config = training_config(estimator=tf_estimator, inputs=inputs)
print(train_config['HyperParameters']['sagemaker_program']) #train.py
assert entry_path == train_config['HyperParameters']['sagemaker_program'] # won't work