requirements.txt in dependencies does not work in TFS container

[harusametime]

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): TensorFlow serving container
• Framework Version:1.12-1.14
• Python Version: 3.6
• CPU or GPU: both
• Python SDK Version: 1.42.1
• Are you using a custom image: No

Describe the problem

Following pre/post-processing section, I want to install pillow for image pre-processing with requirements.txt, and wrote the following code (deploy.py) as explained.

# deploy.py
model = Model(entry_point='inference.py',
              dependencies=['requirements.txt'],
              model_data='s3://mybucket/model.tar.gz',
              role='MySageMakerRole')

deploy.py, inference.py, and requirements.txt are in the same directory.
However no packages are installed. For further investigation, I put the following code at the top of inference.py.

import os
print(os.listdir('/opt/ml/model/code'))
print(os.listdir('/opt/ml/model/code/lib'))

Those returned:

['inference.py', 'lib']
['requirements.txt']

I guess requirements.txt is copied to /opt/ml/model/code/lib and is not used for installing the package; in my opinion, requirements.txt should be in /opt/ml/model/code/.
Please let me know if I missed something.

Minimal repro / logs

Please provide any logs and a bare minimum reproducible test case, as this will be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

• Exact command to reproduce: (same as above)

model = Model(entry_point='inference.py',
              dependencies=['requirements.txt'],
              model_data='s3://mybucket/model.tar.gz',
              role='MySageMakerRole')

[chuyang-deng]

Hi @harusametime, thanks for using SageMaker!

To correctly install the dependencies from requirements.txt, you will need to put the requirements.txt file at the same directory as inference.py and specify the source_dir in Model class.

Elements in dependencies list will be packed into '/opt/ml/model/code/lib' (

sagemaker-python-sdk/src/sagemaker/utils.py

Line 476 in 76d46d0

for dependency in dependencies:

), but our TFS container looks for the requirements file here: https://github.com/aws/sagemaker-tensorflow-serving-container/blob/master/container/sagemaker/serve.py#L32

I apologize for the confusion, and we will update the documentation shortly.

[harusametime]

@ChuyangDeng
Thank you. As you mentioned, the following code worked!

# deploy.py
model = Model(entry_point='inference.py',
              source_dir = './src',
              model_data='s3://mybucket/model.tar.gz',
              role='MySageMakerRole')

File structre

deploy.py
./src
 ├── inference.py
 └── requirements.txt

[Ahtesham00]

I am installing following packages using requirement.txt file in sagemaker training job

requirements.txt file is under code directory

numpy
rouge_score
fire
transformers>=4.26.1
torch
sentencepiece
tokenizers==0.12.1
wandb
git+https://github.com/zphang/transformers@llama_push

Issue I am facing : I am not able to access classes from this specific PR https://github.com/zphang/transformers@llama_push as this should also be installed as others are. I tested this requirements.txt file with my local python virtual env and it was working and installing all the packages including the Specific PR provided above.

What's the issue?

Is SageMaker training job does not install repos using git?

My estimator code is following

estimator = PyTorch(entry_point='train.py',
                    source_dir='code', 
                    role=role[0],
                    framework_version='1.9',
                    py_version='py38',
                    instance_count=1,
                    instance_type='ml.m5.large',
                    input_mode='File')
estimator.fit({'train': train_input})

Please help me regarding this.