You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Cache the transformers model used in ONNX test (pytorch#96793)
Also updating merge_rule to allow ONNX exporter team to update the Docker script by themselves. By default, the model is cached at ~/.cache/huggingface/hub/ under CI jenkins user.
The model is cached so that we don't need to re-download it every time in CI, which causes flaky [CI failures](https://hud.pytorch.org/failure/FAILED%20test%2Fonnx%2Ftest_fx_to_onnx_with_onnxruntime.py%3A%3ATestFxToOnnxWithOnnxRuntime%3A%3Atest_large_scale_exporter_with_tiny_gpt2%20-%20requests.exceptions.ReadTimeout%3A%20HTTPSConnectionPool(host%3D'huggingface.co'%2C%20port%3D443)%3A%20Read%20timed%20out.%20(read%20timeout%3D10.0)).
This is the second part after pytorch#96590
### Testing
Confirm that the model is cached in the Docker image before running the test:
```
jenkins@dd0db85dd34f:~/workspace$ ls -la ~/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/*
/var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/blobs:
total 2460
drwxrwxr-x 2 jenkins jenkins 126 Mar 15 05:48 .
drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 ..
-rw-rw-r-- 1 jenkins jenkins 662 Mar 15 05:48 2c81a6c4c984e95a45338c64a7445c1f0f88077f
-rw-rw-r-- 1 jenkins jenkins 2514146 Mar 15 05:48 b706b24034032bdfe765ded5ab6403d201d295a995b790cb24c74becca5c04e6
/var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/refs:
total 4
drwxrwxr-x 2 jenkins jenkins 18 Mar 15 05:48 .
drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 ..
-rw-rw-r-- 1 jenkins jenkins 40 Mar 15 05:48 main
/var/lib/jenkins/.cache/huggingface/hub/models--sshleifer--tiny-gpt2/snapshots:
total 0
drwxrwxr-x 3 jenkins jenkins 54 Mar 15 05:48 .
drwxrwxr-x 5 jenkins jenkins 48 Mar 15 05:48 ..
drwxrwxr-x 2 jenkins jenkins 50 Mar 15 05:48 5f91d94bd9cd7190a9f3216ff93cd1dd95f2c7be
```
Pull Request resolved: pytorch#96793
Approved by: https://github.com/titaiwangms, https://github.com/ZainRizvi
0 commit comments