[CI] Dockerfile enhancements: cloning

[apostasie]

What is the problem you're trying to solve

Breakout of #4021.
Also see overall Dockerfile analysis for context.

Refresher:

• we clone too much in size (full history), while we do not need it - github clones for a cold-cache run account for about 1G of content
• we clone too often - we should only ever clone any given project once per run

Describe the solution you'd like

Suggesting that:

1. we git clone --quiet (less noise on the logs)
2. we git clone --depth 1 (reduce network traffic and cache size)
3. we separate clone and build - one clone when building multi-arch, and use mounts for build to avoid unnecessary copies (reduce network traffic and cache size)

For reference, one way to do it:

• clone once : https://github.com/farcloser/lepton/blob/main/Dockerfile#L278-L290
• build multi-arch with mount: https://github.com/farcloser/lepton/blob/main/Dockerfile#L534

Note that if we want to keep building using other projects build scripts, we would have to bind mounts RW (as most of these scripts do not allow to build out of tree). I personally believe we should build directly ourselves in most cases instead of relying on Makefiles / scripts (see #4021 for reasons).
If we do not want that, or not feasible in certain cases, we just need a simple lock to prevent cross-platforms builds from conflicting: https://github.com/farcloser/lepton/blob/main/Dockerfile#L753-L755

Let me know if that plan (1, 2, 3) is fine or if you prefer something else.

Additional context

No response

[apostasie]

ADD git:// is available
since 2022 (moby/buildkit#2799) and out of experimental since 2023 (moby/buildkit#3979)

https://docs.docker.com/reference/dockerfile/#adding-files-from-a-git-repository

This might be more cache-efficient when running tests with several containerd versions

(from #4017 (comment))

[apostasie]

ADD git:// is available
since 2022 (moby/buildkit#2799) and out of experimental since 2023 (moby/buildkit#3979)
https://docs.docker.com/reference/dockerfile/#adding-files-from-a-git-repository
This might be more cache-efficient when running tests with several containerd versions

(from #4017 (comment))

After some testing:

• it makes no difference wrt caching (as compared with a regular git clone) - same size, same behavior with changing version ARG for the project being cloned
• code is overly complex (as it needs to accommodate many generic use cases) and hard to reason about (https://github.com/moby/buildkit/blob/master/source/git/source.go#L424)
• what it does (as far as our use case is concerned), is strictly equivalent in result to git clone --depth 1 --branch BRANCH or git clone --depth 1 --revision REV.

On the other hand:

• we can use it without having to install git inside the image (* note: unless we would need it for go mod)
• git --revision is only for recent git which is not available yet in ubuntu / debian - which means we have to do git clone --depth 1 --branch BRANCH AND git rev-parse HEAD instead of just the ADD
• we can probably do away with the git directory itself, which means saving a few megabytes on cache size

I don't quite care either way. Both approach work IMHO.
I have a slight preference for the "manual" way because of two reasons:

1. I find it easier to reason about things when they are explicit / immediately available to review (as the "buildkit" way does hide things away and forces you to go read their source to understand what happens) - but I appreciate that the "buildkit" way is slightly more compact and saves installing git inside the container.
2. it is less entrenched into the specific context of the Dockerfile - if/when we want to move some operations out of the Dockerfile, it is much easier to do without the buildkit-ism

Option 1, the manual way:

ARG VERSION=v2.0.4@1a43cb6a1035441f9aca8f5666a9b3ef9e70ab20
RUN apt-get update -qq; apt-get install -qq --no-install-recommends git
RUN git clone --depth 1 --branch ${VERSION%@*}https://github.com/containerd/containerd.git .; [ "${VERSION#*@}" == "$(git rev-parse HEAD)" ] || { echo "ERROR"; exit 42; }

Option 2, the buildkit way:

ARG VERSION=v2.0.4
ARG REVISION=1a43cb6a1035441f9aca8f5666a9b3ef9e70ab20
ADD https://github.com/containerd/containerd.git#$REVISION .

Note that the buildkit way MAY likely means we have to split out commit shas from the VERSION variable.

Also note this is pseudo-code, not a proper patch - does not account for situations where the revision is empty, etc

Let me know which way you prefer @AkihiroSuda

[AkihiroSuda]

Thanks for testing, the Option 1 seems fine for now, but eventually we should extend BuildKit to support TAG@HASH form

[AkihiroSuda]

https://github.com/containerd/nerdctl/blob/v2.0.4/hack/git-checkout-tag-with-hash.sh is expected to be reusable in other projects that may run git clone outside this script, so the script should not run git clone by default, but it may have an optional flags like --clone=$REPO --shallow

[apostasie]

Ok, so, integrating the script part:

• plan A

ARG VERSION=version@revision
ADD ./hack/git-checkout-tag-with-hash.sh .
RUN git clone --depth 1 --branch ${VERSION%@*}https://github.com/containerd/containerd.git .; ./git-checkout-tag-with-hash.sh "$VERSION"

Pro:
a. smallest change possible
b. keeps a separate /git-checkout-tag-with-hash.sh for other projects to re-use

Cons:
a. /git-checkout-tag-with-hash.sh will do a useless git checkout
b. adds an extra file
c. adds an extra layer
d. some intelligence is moved out of the Dockerfile into the script, making for a (slightly) less intuitive reading / debugging

• plan B

Same as plan A, but addresses the layer concern (c.)

ARG VERSION=version@revision
RUN --mount=target=git-checkout-tag-with-hash.sh,source=./hack/git-checkout-tag-with-hash.sh,type=bind
	git clone --depth 1 --branch ${VERSION%@*}https://github.com/containerd/containerd.git .; ./git-checkout-tag-with-hash.sh "$VERSION"

• plan C

Do away with the script.
Pro:
a. removes the extraneous git checkout
b. everything needed to read/debug in one place
c. removes extra file

ARG VERSION=version@revision
RUN git clone --depth 1 --branch ${VERSION%@*}https://github.com/containerd/containerd.git .; [ "${VERSION%@*}" == "$VERSION" ] || [ "${VERSION#*@}" == "$(git rev-parse HEAD)" ] || { echo "BLABLABLA"; exit 42; }

With some cons:
a. a bit more complex to grok visually, as most RUN commands are less legible than a proper shell script
b. less debugging info / messages in its current form

• plan D

Move the (optional) git clone part to the script.

The script could be smart and figure out if it is already inside a clone or not, and decide to clone if not.

ARG VERSION=version@revision
ARG REPO=blablblabl
RUN --mount=target=git-checkout-tag-with-hash.sh,source=./hack/git-checkout-tag-with-hash.sh,type=bind
	./git-checkout-tag-with-hash.sh "$VERSION" "$REPO"

With script being updated as described.

---

LMK which plan (A, B, C, D) is preferred and I'll send a PR.

[AkihiroSuda]

Thanks, I'd prefer the plan A.
The script can be just ADD-ed to /usr/local/bin in the base stage.

An alternative would be plan E:

RUN git-checkout-tag-with-hash.sh --clone="$REPO" --shallow "$VERSION"

(No need to be smart as in plan D)