Skip to content

Conversation

@atalman
Copy link
Contributor

@atalman atalman commented Nov 18, 2025

Fixes : pytorch/pytorch#159409

We process index.html file rather then whl files. This will allow us to remove all conda and pypi dependencies from our s3 and CDN.

CUDA package example: https://pypi.nvidia.com/nvidia-cuda-nvrtc/
Simple pypi index example: https://pypi.org/simple/filelock/

Deployment strategy:

  1. Deploy only 1 cuda and 1 non cuda package index in nightly
  2. Run delete these packages from nightly repo
  3. Test
  4. Deploy rest of the dependencies
  5. Test

@vercel
Copy link

vercel bot commented Nov 18, 2025

@atalman is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 18, 2025
@atalman atalman requested a review from seemethere November 18, 2025 23:09

def is_nvidia_package(pkg_name: str) -> bool:
"""Check if a package is from NVIDIA and should use pypi.nvidia.com"""
return pkg_name.startswith("nvidia-")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might need to add more logic to this as not all packages start with nvidia-, e.g. https://pypi.org/project/cuda-bindings/ or https://pypi.org/project/cuda-cccl/ in case these will be added in the future. Not blocking for now, just leaving this as a comment for my future self.

Copy link
Contributor

@tinglvv tinglvv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks very much. Just wondering how could we test this change. Seems like we will do it incrementally, so +1ed.

@atalman
Copy link
Contributor Author

atalman commented Nov 19, 2025

HI @tinglvv testing via validation framework. I am currently testing this change

@atalman atalman merged commit 3d54b84 into pytorch:main Nov 20, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[s3_management] Switch dependencies that are on pypi to utilize pypi's CDN

6 participants