Non deterministic helm diff behavior with "three way merge"

[sbugalski]

Hello,

I am using helm diff with Azure DevOps pipelines.

Since recent changes we have issue with three way merge non deterministic helm diff and error exit code.

1. reading three way merge from env prints error
   

Command:

  helm diff upgrade \
    --install $RELEASE_NAME ingress-nginx/ingress-nginx \
    --namespace $RELEASE_NAME \
    --version 4.0.6 \
    --values "values.yaml"

2. Non deterministic behavior related to three way merge from env
   Sometimes same task fails on different pipeline run
   

Same task different results:

Command:

  HELM_DIFF_THREE_WAY_MERGE=false helm diff upgrade \
    --install $RELEASE_NAME ingress-nginx/ingress-nginx \
    --namespace $RELEASE_NAME \
    --version 4.0.6 \
    --values "values.yaml"

[mumoshu]

@sbugalski Hey! Thanks for reporting, but I don't get it yet. Are you basically saying that your pipeline step fails whenever helm-diff prints a message (reading three way merge from env) to stderr?

reading three way merge from env doesn't mean it enabled three-way-merge. It's enabled only when HELM_DIFF_THREE_WAY_MERGE is set to true. It isn't really an error, but a log message.
helm-diff doesn't exist with non-zero code due to that.

helm-diff/cmd/upgrade.go

Line 127 in eb238a1

println("reading three way merge from env")

[mumoshu]

HELM_DIFF_THREE_WAY_MERGE=false doesn't suppress the message, but --three-way-merge does.
In case what you want is to suppress the message, try:

helm diff upgrade \
    --install $RELEASE_NAME ingress-nginx/ingress-nginx \
    --namespace $RELEASE_NAME \
    --version 4.0.6 \
    --values "values.yaml" \
    --three-way-merge=false

But again, it doesn't affect the exit code of the command at all, and the behavior of helm-diff at all, because the default value for --three-way-merge is false. So my guess is that the root cause is in your pipeline step, or perhaps how Azure DevOps pipeline works?

[sbugalski]

Thanks @mumoshu

Wasn't sure how to use the flag, since Docs doesnt say is bool type.

Forgot to say I am using set -euo pipefail. Azure DevOps has strange scripts error handling, but before update it worked. Dunno why it decides to fail on println :D

Anyway, I'll try to set the flag and see if it works. Many thanks for swift reply

[sbugalski]

One more thing. I still can't figure out why it sometimes prints the message and sometimes not

Different clusters but same script is executed...

[mumoshu]

@sbugalski helm-diff always print that message when you omit --three-way-merge flag, which implies that Azure DevOps pipeline sometimes ignores the output to stderr and sometimes not.

[sbugalski]

Many thanks once again. Using flag helped me a lot :)

[mumoshu]

@sbugalski Thanks for confirming! Probably I'll follow up with making logs a bit less verbose, by letting it emit only when the feature is actually enabled via the envvar. I'm not entirely sure how that affects your use-case but I hope it doesn't make things worse.

[mumoshu]

@sbugalski Please see #342. Once that PR is merged and released, you should be able to just omit --three-way-merge=false.