feat: default repack encryption #2821

JGuinegagne · 2022-01-04T23:08:34Z

Issue #, if available:
AppSec finding affecting SageMaker JumpStart TT: V497918765

Description of changes:
This PR changes the default behavior of the SDK when uploading artifacts to the S3 bucket of the Session. Previously, if no kms_key argument was specified, the SDK uploaded artifact to S3 without server-side encryption. Now, the artifact on S3 is uploaded with server-side encryption enabled, and S3 uses the AWS managed KMS key for S3 in the customer account.

According to KMS team and AppSec, there is no downside to enabling server-side encryption in S3 for customers: any S3:getObject call to such objects will be automatically decrypted by S3 if the requester has access to the object. If a customer wishes to disable the server-side encryption for their artifacts anyway (for example to use the public object url), we provide a SessionSettings class as an optional constructor argument that lets them override the default behavior.

This SessionSettings class is meant to be extended to allow customers to parametrize their SageMaker Session.

Testing done:

manually updated the sagemaker package in JumpStart server dependency with matching change, and verified that deploying an endpoint lead to the model tarball in S3 to use server-side encryption with the default AWS managed KMS key for S3.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-01-04T23:15:07Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5c84f70
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-04T23:16:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 54dde1e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-01-04T23:44:36Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.69%. Comparing base (496b259) to head (b863a68).
⚠️ Report is 82 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #2821      +/-   ##
==========================================
+ Coverage   88.67%   88.69%   +0.02%     
==========================================
  Files         173      174       +1     
  Lines       15441    15461      +20     
==========================================
+ Hits        13692    13713      +21     
+ Misses       1749     1748       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sagemaker-bot · 2022-01-04T23:46:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d4c4ed4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:03:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5c84f70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:03:32Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 54dde1e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:18:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 54dde1e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:20:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 54dde1e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:20:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5c84f70
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:26:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d4c4ed4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:34:16Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d4c4ed4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:45:38Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5c84f70
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T00:45:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d4c4ed4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T14:32:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0c6536b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T15:12:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 0c6536b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-05T15:28:10Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 0c6536b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

.gitignore

tests/unit/test_fw_utils.py

sagemaker-bot · 2022-01-06T00:18:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: f6f812c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T00:30:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 15cd67f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T00:53:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: f6f812c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:05:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: f6f812c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:06:52Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: f6f812c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:10:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 15cd67f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:15:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 15cd67f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:24:42Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 15cd67f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T01:37:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 15cd67f
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-06T21:16:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 15cd67f
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-07T19:07:33Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: b863a68
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-07T19:48:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: b863a68
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-07T19:58:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: b863a68
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-01-07T20:34:45Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: b863a68
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Co-authored-by: Shreya Pandit <[email protected]>

JGuinegagne requested review from evakravi and shreyapandit January 4, 2022 23:09

feat: encrypt data in S3 with AWS managed S3 key

54dde1e

JGuinegagne force-pushed the feat/default-repack-encryption branch from 5c84f70 to 54dde1e Compare January 4, 2022 23:12

fix: flake8 failure

d4c4ed4

fix: remove unused mock

0c6536b

JGuinegagne requested a review from mufaddal-rohawala January 5, 2022 18:34

mufaddal-rohawala reviewed Jan 5, 2022

View reviewed changes

.gitignore Show resolved Hide resolved

tests/unit/test_fw_utils.py Outdated Show resolved Hide resolved

chore: rename test case

f6f812c

mufaddal-rohawala approved these changes Jan 6, 2022

View reviewed changes

Merge branch 'dev' into feat/default-repack-encryption

15cd67f

shreyapandit approved these changes Jan 6, 2022

View reviewed changes

evakravi approved these changes Jan 6, 2022

View reviewed changes

Merge branch 'dev' into feat/default-repack-encryption

b863a68

shreyapandit merged commit 8e9d9b7 into aws:dev Jan 7, 2022

ahsan-z-khan pushed a commit to ahsan-z-khan/sagemaker-python-sdk that referenced this pull request Jan 10, 2022

feat: default repack encryption (aws#2821)

d6af831

Co-authored-by: Shreya Pandit <[email protected]>

feat: default repack encryption #2821

feat: default repack encryption #2821

Uh oh!

Conversation

JGuinegagne commented Jan 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Jan 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 4, 2022

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Jan 4, 2022 • edited by codecov bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Jan 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 5, 2022

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Jan 6, 2022

JGuinegagne commented Jan 4, 2022 •

edited

Loading

codecov-commenter commented Jan 4, 2022 •

edited by codecov bot

Loading