helm diff failes when CRDs aren't present

[czunker]

helm-diff version: 3.1.0
helm version: 3.1.0
helmfile version: 0.94.1 or 0.100.0

Hi,

we have problems installing prometheus-operator with helmfile. We get these error messages:

failed processing release prometheus-operator: helm exited with status 1:
Error: Failed to render chart: exit status 1: manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "Alertmanager" in version "monitoring.coreos.com/v1", unable to recognize no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "Prometheus" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"]
Error: plugin "diff" exited with error

I could pin it down to helm-diff:

$ helm diff upgrade --reset-values --allow-unreleased prometheus-operator stable/prometheus-operator --namespace monitoring --values /tmp/prom-op.values --detailed-exitcode
Error: Failed to render chart: exit status 1: manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "Alertmanager" in version "monitoring.coreos.com/v1"

 Error: plugin "diff" exited with error

I installed prometheus-operator with helm-diff v3.0.0-rc.7. Then I manually deleted the alertmanager CRD and executed only helm diff.

v3.0.0-rc.7 doesn't show any diff. v3.1.0 exists with the above error.
When the CRDs are present, the diff command works without problems.

[diversario]

Seeing this with Helm 3.1.2, helm-diff 3.1.1, Helmfile 0.106.2. This seems to be generally CRD-related:

Comparing release=private-system, chart=private/private-system
Comparing release=dashboard, chart=stable/kubernetes-dashboard
Comparing release=node-problem-detector, chart=stable/node-problem-detector
Comparing release=metrics-server, chart=stable/metrics-server
Comparing release=kube2iam, chart=stable/kube2iam
Comparing release=cert-manager, chart=jetstack/cert-manager
Comparing release=dns-internal, chart=stable/external-dns
Comparing release=ngx-ingress, chart=stable/nginx-ingress
Comparing release=kubewatch, chart=stable/kubewatch
Comparing release=cluster-autoscaler, chart=stable/cluster-autoscaler
Comparing release=weave-scope, chart=stable/weave-scope
Comparing release=fluentd-aggregator, chart=private/fluentd-aggregator
Comparing release=fluentd-collector, chart=private/fluentd-collector
Comparing release=prometheus-operator, chart=stable/prometheus-operator
Comparing release=grafana-dashboards, chart=private/grafana-dashboards
Comparing release=cilium, chart=private/cilium
Comparing release=hubble, chart=private/hubble
Comparing release=goldilocks, chart=fairwinds-stable/goldilocks
in k8s.testing.ue1.private.io.yaml: in .helmfiles[0]: in ./core.yaml: 7 errors:
err 0: failed processing release private-system: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "CiliumNetworkPolicy" in version "cilium.io/v2", unable to recognize "": no matches for kind "ClusterIssuer" in version "cert-manager.io/v1alpha2", unable to recognize "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1"]
  
  Error: plugin "diff" exited with error
err 1: failed processing release kube2iam: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
  
  Error: plugin "diff" exited with error
err 2: failed processing release ngx-ingress: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
  
  Error: plugin "diff" exited with error
err 3: failed processing release fluentd-aggregator: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
  
  Error: plugin "diff" exited with error
err 4: failed processing release fluentd-collector: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
  
  Error: plugin "diff" exited with error
err 5: failed processing release prometheus-operator: helm exited with status 1:
  Error: Failed to render chart: exit status 1: manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
  manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
  manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
  manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
  manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
  Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "Alertmanager" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "Prometheus" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "PrometheusRule" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"]
  
  Error: plugin "diff" exited with error
err 6: failed processing release cilium: helm exited with status 1:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
  
  Error: plugin "diff" exited with error
Command exited with code 1.

prometheus-operator chart version is 8.9.2. This wasn't happening with 3.0.0-rc.7.

[mmazek]

We run into the same issue.
Helm v3.2.0
Helm diff v3.1.1
Helmfile v0.111.0

We're not able to do fresh install of any Helm Chart with CRD in it.

[mumoshu]

manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"

Seems like the prometheus-operator chart doesn't support the Helm-3 way to handle CRDs yet?

I'd suggest reading helm/helm#6243 and checking if the chart does support it.

[kyounger]

@mumoshu even with charts that use the helm3 crds/ directory, this is still an issue. Diff tries to validate against an API that just doesn't exist in the server yet. Not sure if there's much that can be done here other than maybe giving an option to avoid the validation, or detecting that the CRD isn't installed and optionally skipping validation for resources that it [clearly] can't validate against the uninstalled CRD.

If the CRD isn't installed, then technically the diff would be all "green" any resource of that kind anyway, I think.

[mumoshu]

@kyounger I think I've finally got this issue correctly. #228 should fix it!