cmd/go: use shallow clones for new git checkouts

[mengzhuo]

Currently go get * will download all changes data of repositories.
We can improve the download waiting time/size by passing depth argument.

I have already submit a code-review for this at
https://go-review.googlesource.com/#/c/16360/
as @bradfitz asked.

[rakyll]

Some users depend on go get to clone the repos. How is this going to affect them?

[garukun]

+1 for the CL. Maybe add go clone to support @rakyll's case?

[ALTree]

A suggestion: some data about the magnitude of the improvement in the size of the downloaded repository could probably help the reviewers.

[bradfitz]

It's already the case that "go get" won't always fetch the full history in some cases (subversion), if that precedent matters.

I'm also in favor or switching to shallow clones. Personal motivation: Camlistore's Git repo is huge, but mostly because we used to have some huge assets and other libraries in it, since gone. But that means for a user just wanting a few files, they have to download megabytes.

People who want history can go into the git directories and run the git commands.

Or we can add a new -full or -deep flag to go get.

[bradfitz]

/cc @adg @rsc

[fsouza]

+1 for the feature and for @bradfitz's -full flag, although I think users can always unshallow their repositories manually running git fetch --unshallow.

[adg]

I do think it would be worth switching to shallow clones.

I don't see much value in adding a flag to the go tool.

[fsouza]

Maybe having an environment variable for the git command (as well as other vcs commands) would do the trick and make it more flexible, if that's the case. Just like git allows customizing the ssh command with the GIT_SSH_COMMAND variable. Probably off-topic, though.

[davecheney]

I think that these requests are outside the remit of go get.

We all know go get is just a wrapper around git clone. As time goes on, a few little tweaks have been added to the clone model, and the result is more requests for more tweaks, more flags, etc.

This is natural and expected, go get is a simplified wrapper around a more complicated command so it is a logical conclusion that go get itself would have to become complex in turn.

Rather than this outcome, I suggest that we document the limitations of go get and recommend to people who need more that they simply skip go get, and use git clone; giving them the full power of the tool for their specific scenario without having to further complicate the simple model that go get provides.

[mengzhuo]

@davecheney
I agree that we should think about what "go get" for at the beginning.
IMHO "get" means "get the right source code" for build/compile instead of "get the whole repository for investigation/study" and git --depth arguments making "go get" do the right thing.

[rakyll]

For the sake of clarity...

Even though, it is not the core job of the go tool, I raised to issue of being able to deep copy because of the canonical URLs.

I simply want to be able to run

go get -d golang.org/x/tools/...

rather than figuring out what exact repo it is pointing to. But I simply can run git fetch --unshallow once it is cloned shallowly.

[omeid]

+1 shallow clone.

I just tried to do go get github.com/openshift/origin/cmd/oc and after a very long wait I ended up selectively shallow cloning docker/docker, kubernetes/kubernetes, openshift/origin and couple of other dependencies.

[mengzhuo]

@davecheney
@adg

The title is miss leading.
It should be:
cmd/go: improve git pull download size and speed (by adding --depth arguments into git VCS command)
It doesn't add any params into "go get" cmd
Sorry for my poor English (sigh)