xet protocol specification in hub docs #1952
Conversation
There was a problem hiding this comment.
i think it would maybe make sense to nest under https://huggingface.co/docs/hub, but what do you think?
That works for me. I was initially going to the path Still also thinking where to link into the xet spec out of the other docs (storage-backends page 👍 , but anything else TBD) |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Is it at all possible to replace the table of contents left-side sidebar on the I don't think it makes sense to add all the new pages to the sidebar on all of the hub documentation pages, but while on the specification I do think users should be able to navigate between all the pages. |
assafvayner
changed the title
docs/hub/storage-backends.md
Outdated
|
|
||
| This leads to a worse developer experience along with a proliferation of additional storage. | ||
|
|
||
| ## Open Source Xet Protocol |
There was a problem hiding this comment.
This should be moved up to BEFORE the Backwards Compatibility with LFS section. Putting at the bottom of the page buries its discoverability. The flow of the page is 'overview, usage, protocol, backwards compat, security, legacy'
docs/hub/storage-backends.md
Outdated
| The primary limitation of Git LFS is its file-centric approach to deduplication. Any change to a file, irrespective of how large of small that change is, means the entire file is versioned - incurring significant overheads in file transfers as the entire file is uploaded (if committing to a repository) or downloaded (if pulling the latest version to your machine). | ||
|
|
||
| This leads to a worse developer experience along with a proliferation of additional storage. | ||
|
|
There was a problem hiding this comment.
Earlier section (not sure how to get comment over to area not in diff, sigh) - Security Model: I think you can deep link into protocol to refer to areas of the protocol that speak to the privacy preserving global dedup.
There was a problem hiding this comment.
There's 2 things to link to in regards to "security model"
A. in deduplication.md the very short section titled #### HMAC Security Mechanism is relevant to what you described.
B. the auth.md file as a whole if it's desired to show how the hub is the source of truth on auth for CAS
There was a problem hiding this comment.
Sweet! Just did a first pass mostly focused on little nits/inconsistencies. Planning on doing another pass tomorrow.
I agree that navigation is something to address - @rajatarya's approach in #1956 seems promising.
Do we agree that "Xet" should be uppercased in all instances?
Porting xet specification to final location in hub docs (from xet-core github).
Includes complete specification for xet protocol.