Gracefully handle semantically meaningless changes

[Eh2406]

In rust-lang/crates.io#5066 there are plans for the official index to be normalized to match the crates.io database. This unfortunately creates a lot of churn in the index. By churn I mean sorting the entries by version number, sorting the features, sorting the deps, etc. None of this should be semantically meaningful, but may lead to weird bugs in this crate.

This issue is to make sure that this crate handles these changes gracefully, hopefully before the index normalization is actually performed.

[Byron]

Thanks for the heads-up! I will be taking a stab at this today and will be using https://github.com/arlosi/crates.io-index as a basis to create a test-case from will hopefully be enough to make it able to cope with this.

Something I was wondering though: What if docs.rs would regenerate each version of each crate with a priority of -2? Would that not lead to nicer docs as they are now created by a much more recent rustdoc? If docs.rs had such capability, I could imagine it doing this once a year or so to normalize the looks of docs. There are of course a lot of arguments against that, number one coming to my mind being that fewest people presumably look at old version of the docs anyway.

[Eh2406]

That is a really interesting suggestion for the docs.rs team. I don't see how it's relevant to this attempt by the crates.io team too bring there two different databases In Sync with each other. What did I miss?

[Byron]

Probably it isn't, after all rebuilding everything can be achieved more easily than abusing bugs I thought crates-index-diff would exhibit. That was wrong though, having looked at the code I believe it would probably just go horribly wrong :D.

The reason I really like this task is that…

• …the test suite can be overhauled with git-testtools in tow
• …gitoxide was always lacking line-diffing capabilities and now is a chance to build a trial version of this
• …using gitoxide for diffing will definitely be a speedup

So in the end, git2 will only be used for fetching, and the diff logic will be rebuilt to be agnostic to meaningless changes. Let's see how it pans out.