Feature: SageMaker Remote Function #3797

nmadan · 2023-04-18T23:59:34Z

Issue #, if available:

Description of changes: support running python functions as training jobs by using @Remote decorator
Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2023-04-19T00:01:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 356bae5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T00:01:28Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 356bae5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T00:01:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 356bae5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T00:02:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 356bae5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T00:34:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e1557c0
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T01:14:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 17d5763
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T01:53:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 17d5763
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T01:58:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 17d5763
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T02:19:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 17d5763
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T02:33:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 17d5763
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T05:53:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: aab1aae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T06:34:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: aab1aae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T06:37:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: aab1aae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T06:41:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: aab1aae
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T06:45:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: aab1aae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T07:07:25Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: aab1aae
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2023-04-19T07:10:12Z

Codecov Report

Merging #3797 (aab1aae) into master (13bb3c9) will decrease coverage by 0.72%.
The diff coverage is 88.20%.

@@            Coverage Diff             @@
##           master    #3797      +/-   ##
==========================================
- Coverage   89.86%   89.14%   -0.72%     
==========================================
  Files        1016      246     -770     
  Lines       96416    24938   -71478     
==========================================
- Hits        86644    22232   -64412     
+ Misses       9772     2706    -7066

Impacted Files	Coverage Δ
src/sagemaker/feature_store/feature_group.py	`76.82% <ø> (ø)`
src/sagemaker/model_monitor/model_monitoring.py	`69.60% <56.25%> (ø)`
src/sagemaker/s3.py	`79.16% <60.00%> (ø)`
src/sagemaker/session.py	`78.22% <71.42%> (ø)`
src/sagemaker/remote_function/invoke_function.py	`76.47% <76.47%> (ø)`
...runtime_environment/runtime_environment_manager.py	`79.75% <79.75%> (ø)`
src/sagemaker/local/local_session.py	`83.25% <80.00%> (ø)`
...ntime_environment/bootstrap_runtime_environment.py	`85.07% <85.07%> (ø)`
src/sagemaker/remote_function/job.py	`88.73% <88.73%> (ø)`
...rc/sagemaker/remote_function/core/serialization.py	`96.20% <96.20%> (ø)`
... and 9 more

... and 1243 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

trajanikant

\bot run unit-tests, local

Co-authored-by: Ao Guo <[email protected]> Co-authored-by: Rohan Gujarathi <[email protected]> Co-authored-by: Zhankui Lu <[email protected]> Co-authored-by: Dipankar Patro <[email protected]> Co-authored-by: Mourya Baddam <[email protected]> Co-authored-by: Namrata Madan <[email protected]>

sagemaker-bot · 2023-04-19T17:38:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T17:40:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1689c7c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T17:42:05Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T17:44:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T17:50:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T20:54:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1689c7c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

trajanikant

/bot run slow-tests

sagemaker-bot · 2023-04-19T22:15:44Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T22:15:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T22:15:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T22:15:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2023-04-19T23:19:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1689c7c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

nmadan requested a review from a team as a code owner April 18, 2023 23:59

nmadan requested review from trajanikant and removed request for a team April 18, 2023 23:59

nmadan force-pushed the remote_function branch from 356bae5 to e1557c0 Compare April 19, 2023 00:29

trajanikant force-pushed the remote_function branch from e1557c0 to 17d5763 Compare April 19, 2023 01:10

nmadan force-pushed the remote_function branch from 17d5763 to aab1aae Compare April 19, 2023 05:50

trajanikant reviewed Apr 19, 2023

View reviewed changes

trajanikant force-pushed the remote_function branch from aab1aae to 1689c7c Compare April 19, 2023 16:45

trajanikant reviewed Apr 19, 2023

View reviewed changes

trajanikant changed the title ~~feature: sagemaker remote function~~ Feature: SageMaker Remote Function Apr 19, 2023

trajanikant merged commit 46215ae into aws:master Apr 19, 2023

Feature: SageMaker Remote Function #3797

Feature: SageMaker Remote Function #3797

Uh oh!

Conversation

nmadan commented Apr 18, 2023

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Apr 19, 2023

Codecov Report

Uh oh!

trajanikant left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Apr 19, 2023

trajanikant left a comment •

edited

Loading