Commit 1d7ab49
New Folder Structure Implementation (#4726)
* Update SMP v2 notebooks to use latest PyTorch 2.3.1, TSM 2.4.0 release (#4678)
* Update SMP v2 notebooks to use latest PT2.3.1-TSM2.4.0 release.
* Update SMP v2 shared_scripts
* Update minimum sagemaker pysdk version to 2.224
* Parsash2 patch 1 (#4690)
* tutorials-after-initial-feedback
Added descriptive text to make the notebooks stand on their own.
* move athena notebook into dedicated folder
* renamed athena end2end notebooks
* moved pyspark notebook into dedicated directory
* minor change: consistent directory naming convention
* Added overview, headers, and explantory text
Tested the notebook end to end. Added more context for processing jobs and cleaning up. The output is visible in the cells.
* Added overview, headers, explanatory text
Also added troubleshooting note from further testing.
* fix directory locations for new notebooks
* clear notebook outputs
* added integration for ci test results
* updated formatting with black-nb
* update athena notebook: fix parse predictions
* fixed ci integration for pyspark-etl-training notebook
---------
Co-authored-by: Janosch Woschitz <[email protected]>
* MLflow setup (#4689)
* Add SageMaker MLflow examples
* Add badges
* Add MLflow setup notebook; upgrade SageMaker Python SDK for deployment notebook
* Linting
* More linting changes
---------
Co-authored-by: Bobby Lindsey <[email protected]>
* feat: Model monitor json support for Explainability and Bias (#4696)
* initial commit of Blog content: "using step decorator for bedrock fine tuning" (https://sim.amazon.com/issues/ML-16440) (#4657)
* initial commit of using step decorator for bedrock fine tuning
* ran black command on the notebook
* Added CI badges
* Added CI badges
* fixed typo in notebook title
---------
Co-authored-by: Ashish Rawat <[email protected]>
Co-authored-by: Zhaoqi <[email protected]>
* New folder structure (#4694)
* Deleted 17 duplicate notebooks (#4685)
* Updated README, removed broken links and fixed markdown (#4687)
* New Folder Structure Implementation - Archived remaining geospatial example notebooks (#4691)
* Archived remaining geospatial example notebooks
* Removed geospatial from README.md
* Archived remaining workshop notebooks (#4692)
* Archived outdated example notebooks between 1-90 views (#4693)
---------
Co-authored-by: jsmul <[email protected]>
* Revert "New folder structure (#4694)" (#4701)
This reverts commit 970d88e due to broken blog links
* adding notebook for forecast to canvas workshop (#4704)
* adding notebook for forecast to canvas workshop
* formatting the notebook using black
* Adds notebook for deploying and monitoring llm on sagemaker usin fmeval for evaluation (#4705)
Co-authored-by: Brent Friedman <[email protected]>
* Update athena_ml_workflow_end_to_end.ipynb (#4702)
Fixed bucket names and external links. No change to underlying code or formatting.
Co-authored-by: sage-maker <[email protected]>
* SageMaker FasterAutoscaling Llama3-8B TGI, real-time endpoints (#4712)
* SageMaker FasterAutoscaling Llama3-8B TGI, real-time endpoints
* Moved trigger autoscaling to shell script. Removed shell=True in subprocess.Popen
---------
Co-authored-by: Aditi Sharma <[email protected]>
* Removed bug where automatic permission attachment errors (#4714)
* new file structure: re-added pull request template and files needed for rtd build
* SageMaker pipeline step decorator blog (#4715)
* initial commit of using step decorator for bedrock fine tuning
* ran black command on the notebook
* Added CI badges
* Added CI badges
* fixed typo in notebook title
* added comments and reviewer feedback
---------
Co-authored-by: Ashish Rawat <[email protected]>
Co-authored-by: Zhaoqi <[email protected]>
* new file structure: re-added RTD files, finished archiving/organizing, added READMEs to folders
* new file structure: updated index.rst with new file structure / archived one notebook, renamed one notebook
* new file structure: updated contributing.md
* LLM Monitoring Multiple Evals (#4718)
* Removed bug where automatic permission attachment errors
* Adds notebook for monitoring llm with multiple eval libraries
---------
Co-authored-by: Brent Friedman <[email protected]>
Co-authored-by: nileshvd <[email protected]>
* new file structure: updated badging path for end-to-end examples
* new file structure: updated badging path for prepare data notebooks
* new file structure: updated badging path for build and train models folder
* new file structure: updated badging path for deploy and monitor folder
* new file structure: updated badging path for generative ai folder
* new file structure: updated badging path for ml ops folder
* new file structure: updated badging path for responsible ai folder
* new file structure: updated folder READMEs, and added several notebook titles
* new folder structure: updated pull request template
* new file structure: updated readme with announcement, added new notebook naming excel sheet
---------
Co-authored-by: Victor Zhu <[email protected]>
Co-authored-by: parsash2 <[email protected]>
Co-authored-by: Janosch Woschitz <[email protected]>
Co-authored-by: Bobby Lindsey <[email protected]>
Co-authored-by: Bobby Lindsey <[email protected]>
Co-authored-by: zicanl-amazon <[email protected]>
Co-authored-by: ashrawat <[email protected]>
Co-authored-by: Ashish Rawat <[email protected]>
Co-authored-by: Zhaoqi <[email protected]>
Co-authored-by: pro-biswa <[email protected]>
Co-authored-by: brentfriedman725 <[email protected]>
Co-authored-by: Brent Friedman <[email protected]>
Co-authored-by: sage-maker <[email protected]>
Co-authored-by: Praveen Chamarthi <[email protected]>
Co-authored-by: Aditi Sharma <[email protected]>
Co-authored-by: nileshvd <[email protected]>1 parent e5a0e78 commit 1d7ab49
File tree
1,043 files changed
+10643
-6557
lines changed- end_to_end_ml_lifecycle
- prepare_data
- sm-data_wrangler_data_prep_widget
- sm-feature_store_feature_processor
- sm-feature_store_ground_truth_classification_output_to_store
- sm-feature_store_introduction
- sm-ground_truth_active_learning_workflow_bring_your_own_model
- sm-ground_truth_annotation_consolidation_image_classification
- sm-ground_truth_object_detection_example
- sm-ground_truth_video_quality_metrics
- sm-ingest_data_with_athena/image
- sm-ingest_data_with_redshift/image
- sm-marketplace_augmented_ai_with_marketplace_ml_models
- img
- src
- sm-processing_feature_transformation_with_dask
- sm-processing_spark_processing
- sm-spark_pca_kmeans
- img
- build_and_train_models
- sm-cnn_pytorch
- source
- sm-distributed_data_parallelism_pytorch
- sm-distributed_model_parallel
- sm-experiments_job_tracking_script_mode_traning_job_pytorch/images
- sm-forecast_deepar_time_series_modeling
- sm-fsdp_finetuning_of_llama_v2
- sm-heterogeneous_clusters_for_model_training
- sm-heterogeneous_clusters_training
- sm-hyperparameter_tuning_pytorch
- sm-inference_script_mode_xgboost
- images
- sm-introduction_to_blazingtext_word2vec_text8
- sm-introduction_to_ip_insights
- sm-introduction_to_lda
- sm-introduction_to_ntm
- sm-introduction_to_object2vec_sentence_similarity
- sm-jax_bring_your_own
- sm-managed_spot_training_xgboost
- sm-marketplace_build_model_package_for_listing
- sm-marketplace_building_your_own_container_as_package
- sm-name_entity_recognition_document
- containers
- entity_recognition
- finetuning
- question_answering
- relationship_extraction
- package
- data
- models
- summarization
- sm-object_detection_birds
- tools
- sm-random_cut_forest_example
- sm-regression_xgboost
- sm-remote_function_pytorch_mnist
- sm-remote_function_quick_start
- sm-scikit_build_your_own_container
- sm-script_mode_distributed_training_horovod_tensorflow
- sm-semantic_segmentation
- sm-smddp_bert
- sm-training_compiler_language_modeling_multi_gpu_multi_node
- sm-training_compliler_single_node_single_gpu_bert
- scripts
- deploy_and_monitor
- sm-a_b_testing
- sm-app_autoscaling_realtime_endpoints_inference_components
- utils
- sm-app_autoscaling_realtime_endpoints_step_scaling
- utils
- sm-app_autoscaling_realtime_endpoints
- utils
- sm-async_inference_walkthrough
- sm-async_inference_with_python_sdk
- sm-batch_inference_with_torchserve
- sm-batch_transform_pca_dbscan_movie_clusters
- sm-batch_transform_pytorch
- sm-batch_transform_with_torchserve
- sm-clarify_model_bias_monitor_batch_transform_json
- sm-clarify_model_bias_monitor_batch_transform
- sm-clarify_model_bias_monitor_for_endpoint_json
- sm-clarify_model_bias_monitor_for_endpoint
- sm-clarify_model_explainability_monitor_attribution_drift_for_endpoint_json/test_data
- sm-clarify_model_monitor_model_feature_attribution_drift_for_endpoint/model
- sm-deploy_trained_pytorch_model
- code
- utils
- sm-deployment_guardrails_update_inference_endpoint_with_linear_traffic_shifting
- sm-deployment_guardrails_update_inference_endpoint_with_rolling_deployment
- sm-deployment_guardrails_update_inference_endpoint_with_with_canary_traffic_shifting
- sm-experiments_local_experiment_tracking/images
- sm-host_pretrained_model_bert
- sm-inference_pipeline_with_scikit_linear_learner
- sm-lineage_cross_account_queries_with_ram
- sm-marketplace_using_model_package_arn
- sm-mme_with_torchserve
- sm-model_monitor_batch_transform_data_quality_on_schedule
- model
- test_data
- sm-model_monitor_batch_transform_model_quality_on_schedule
- sm-model_monitor_bias_and_explainability_monitoring
- sm-model_monitor_byoc_llm_monitor
- src
- components
- utils
- sm-model_monitor_introduction
- sm-model_monitor_model_quality_monitoring
- sm-multi_container_endpoint_direct_invocation
- sm-multi_model_endpoint_bring_your_own_container
- sm-serverless_inference_huggingface_text_classification
- sm-shadow_variant_shadow_api
- sm-triton_inferentia2
- sm-triton_mme_bert_trt
- sm-triton_mme_gpu_ensemble_dali
- sm-triton_nlp_bert
- sm-triton_realtime_sme_flan_t5
- sm-triton_tensorrt-sentence_transformer
- sm-xgboost_bring_your_own_model
- generative_ai
- sm-finetuning_huggingface_with_your_own_scripts_and_data
- sm-mixtral_8x7b_fine_tune_and_deploy
- ml_ops
- sm-ml_lineage_tracking_model_governance_graph
- sm-mlflow_deployment
- sm-mlflow_hpo
- sm-mlflow_pipelines
- sm-mlflow_setup
- sm-mlflow_training
- sm-model_monitor_batch_transform_data_quality_with_pipelines_on_demand
- sm-model_monitor_batch_transform_model_quality_with_pipelines_on_demand
- sm-pipelines_batch_inference_step_decorator
- sm-pipelines_callback_step
- sm-pipelines_clarify_model_monitor_integration
- sm-pipelines_emr_step_using_step_decorator
- sm-pipelines_lambda_step
- sm-pipelines_launching_autopilot_with_auto_ml_step
- sm-pipelines_local_mode
- sm-pipelines_preprocess_train_evaluate_batch_transform
- sm-pipelines_selective_execution
- sm-pipelines_step_caching
- sm-pipelines_step_decorator_with_classic_training_step
- sm-pipelines_step_decorator_with_condition_step
- sm-pipelines_step_decorator
- responsible_ai
- sm-autopilot_model_explanation_with_shap
- sm-clarify_explainability_image_classification
- sm-clarify_natural_language_processing_online_explainability
- sm-clarify_object_detection
- sm-clarify_online_explainability_mme_xgboost
- sm-clarify_online_explainability_tabular
- sm-clarify_text_explainability_text_sentiment_analysis
- sm-clarify_time_series_bring_your_own_model
- sm-model_governance_model_card_with_model_package
- sm-model_governance_model_card
- archived
- albert-base-v2
- images
- scripts
- athena_ml_workflow_end_to_end
- auto-scaling
- autogluon-tabular-containers
- config
- scripts
- autopilot-serverless-inference
- images
- bandits_statlog_vw_customEnv
- sim_app
- src
- basic-training-container
- docker
- code
- notebook
- scripts
- bedrock-examples
- credit_card_fraud_detector
- lambda/model-invocation
- src/package
- data_parallel_bert
- dw_flow
- fairness_and_explainability_json_format
- analysis_config
- container
- recordings
- fairness_and_explainability_jsonlines
- model
- test_data
- fairness_and_explainability_json
- model
- test_data
- fairness_and_explainability_spark
- fairness_and_explainability
- model
- test_data
- falcon
- framework-container
- docker
- code
- images
- package
- src/custom_framework_training
- scripts
- source_dir
- frameworks-tensorflow
- code
- fully_sharded_data_parallel-falcon
- scripts
- getting_started
- data
- images
- huggingface-inference-recommender
- code
- sample-payload
- huggingface_sentiment_classification
- scripts
- huggingface_text_classification
- identify_key_insights_from_textual_document
- implicit_bpr
- data
- images
- ingest-with-aws-services
- image
- llm_monitor_byoc
- data
- src
- components
- utils
- local_experiment_tracking
- images
- mme-on-gpu
- images
- workspace
- mnist
- code
- model_monitor_tensorflow
- model
- multi_modal_parallel_sagemaker_labeling_workflows_with_step_functions
- cloudformation
- deploy
- src
- boto3_layer
- datalake/scripts
- lambda_src
- api_batch_create
- api_batch_metadata_post
- api_batch_show
- api_workforce_show
- labeling_job_state_change
- step_functions_batch_error
- step_functions_copy_logs_and_send_batch_completed
- step_functions_send_second_level_sns_and_check_response
- step_functions_transformation
- step_functions_trigger_labeling_job
- step_functions_wait_batch_completion
- step_functions_wait_for_metadata_supply
- lambda_test
- api_batch_create
- api_batch_show
- labeling_job_state_change
- step_function_batch_error
- step_functions_copy_logs_and_send_batch_completed
- step_functions_send_second_level_sns_and_check_response
- multi_model_catboost
- container
- mme_catboost
- data
- mme_catboost
- models
- mme_catboost
- multi_model_pytorch
- code
- multicategory_sec
- code
- multimodal_tabtext
- code
- notebook-job-step
- onnx-roberta-backend
- config_templates
- utils
- preprocessing-audio-data-using-a-machine-learning-model
- img
- pyspark_mnist
- img
- python-sdk
- code
- pytorch-ic-model
- output
- pytorch-sagemaker-huggingface
- files
- scripts
- preprocess
- train
- pytorch_cnn_cifar10
- pytorch_extend_container_train_deploy_bertopic
- container
- bert-topic-byoreq
- bert-topic
- utils
- pytorch_yolov5_training_and_hpo
- docker
- yolov5
- pytorch
- code
- utils
- r_serving_with_fastapi
- r_serving_with_plumber
- resnet50
- triton-serve-pt/resnet
- triton-serve-trt/resnet
- workspace
- resnet_onnx_pytorch_tensorRT-backend
- images
- workspace
- resnet_pytorch_python-backend
- images
- triton-serve-pt
- workspace
- rl_gamerserver_ray
- ap-client
- autopilot-client-image
- specs
- ap-server
- .chalice
- deployed
- common
- sagemaker_rl
- orchestrator
- clients/ddb
- exceptions
- utils
- workflow
- datatypes
- manager
- tests
- unit
- game-env
- minecraft-server-image
- specs
- spot-sig-handler-image
- src
- rl_traveling_salesman_vehicle_routing_coach
- src
- rl_unity_ray
- src
- roberta-base
- scripts
- sagemaker-autopilot-pipelines
- sagemaker-featurestore
- data
- images
- sagemaker-pipeline-compare-model-versions
- dataset
- large
- small
- test
- images
- sagemaker-pipeline-parameterization
- credit_risk
- customer_churn
- images
- sagemaker-script-mode
- my_custom_library
- pytorch_script
- scikitlearn_script
- xgboost_script
- sagemaker_job_tracking
- scikit_learn_bring_your_own_model
- code
- scikit_learn_data_processing_and_model_evaluation
- code
- scikit_learn_iris
- sentiment_parallel_batch
- scripts
- shadow-console
- images
- single_gpu_single_node
- scripts
- sklearn-inference-recommender
- code
- sklearn
- sme_resnet_pytorch_python-backend
- triton-serve-pt/resnet
- workspace
- stable_diffusion
- model_repository
- pipeline
- 1
- text_encoder
- vae
- studio-scheduling
- t5_pytorch_python-backend
- workspace
- tensorflow-cloudwatch
- code
- tensorflow2-california-housing-sagemaker-pipelines-deploy-endpoint
- code
- images
- tensorflow2_mnist
- code
- tensorflow_moving_from_framework_mode_to_script_mode
- tensorflow_script_mode_training_and_serving
- tensorflow_single_gpu_single_node
- scripts
- tensort-rt
- workspace
- text-to-image-fine-tuning
- code
- config
- using_step_decorator_with_selective_execution
- visual_object_detection
- src
- visualization
- xgboost_abalone
- images
- xgboost_bring_your_own
- xgboost
- .github
- ISSUE_TEMPLATE
- _static
- _templates
- still_to_organize
- advanced_functionality/custom-training-containers
- reinforcement_learning
- rl_game_server_autopilot
- sagemaker
- sagemaker-pipelines
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
1,043 files changed
+10643
-6557
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
Lines changed: 16 additions & 18 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | 12 | | |
14 | 13 | | |
15 | 14 | | |
16 | 15 | | |
17 | 16 | | |
18 | 17 | | |
19 | 18 | | |
20 | | - | |
| 19 | + | |
21 | 20 | | |
22 | 21 | | |
23 | 22 | | |
| |||
1256 | 1255 | | |
1257 | 1256 | | |
1258 | 1257 | | |
1259 | | - | |
1260 | 1258 | | |
1261 | 1259 | | |
1262 | 1260 | | |
1263 | 1261 | | |
1264 | 1262 | | |
1265 | 1263 | | |
1266 | 1264 | | |
1267 | | - | |
| 1265 | + | |
1268 | 1266 | | |
1269 | | - | |
| 1267 | + | |
1270 | 1268 | | |
1271 | | - | |
| 1269 | + | |
1272 | 1270 | | |
1273 | | - | |
| 1271 | + | |
1274 | 1272 | | |
1275 | | - | |
| 1273 | + | |
1276 | 1274 | | |
1277 | | - | |
| 1275 | + | |
1278 | 1276 | | |
1279 | | - | |
| 1277 | + | |
1280 | 1278 | | |
1281 | | - | |
| 1279 | + | |
1282 | 1280 | | |
1283 | | - | |
| 1281 | + | |
1284 | 1282 | | |
1285 | | - | |
| 1283 | + | |
1286 | 1284 | | |
1287 | | - | |
| 1285 | + | |
1288 | 1286 | | |
1289 | | - | |
| 1287 | + | |
1290 | 1288 | | |
1291 | | - | |
| 1289 | + | |
1292 | 1290 | | |
1293 | | - | |
| 1291 | + | |
1294 | 1292 | | |
1295 | | - | |
| 1293 | + | |
1296 | 1294 | | |
1297 | 1295 | | |
1298 | 1296 | | |
| |||
Lines changed: 16 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| |||
947 | 947 | | |
948 | 948 | | |
949 | 949 | | |
950 | | - | |
| 950 | + | |
951 | 951 | | |
952 | | - | |
| 952 | + | |
953 | 953 | | |
954 | | - | |
| 954 | + | |
955 | 955 | | |
956 | | - | |
| 956 | + | |
957 | 957 | | |
958 | | - | |
| 958 | + | |
959 | 959 | | |
960 | | - | |
| 960 | + | |
961 | 961 | | |
962 | | - | |
| 962 | + | |
963 | 963 | | |
964 | | - | |
| 964 | + | |
965 | 965 | | |
966 | | - | |
| 966 | + | |
967 | 967 | | |
968 | | - | |
| 968 | + | |
969 | 969 | | |
970 | | - | |
| 970 | + | |
971 | 971 | | |
972 | | - | |
| 972 | + | |
973 | 973 | | |
974 | | - | |
| 974 | + | |
975 | 975 | | |
976 | | - | |
| 976 | + | |
977 | 977 | | |
978 | | - | |
| 978 | + | |
979 | 979 | | |
980 | 980 | | |
981 | 981 | | |
| |||
0 commit comments