[obs] Refactor alerts for image builds

For the last 30 days:

GitpodImagebuildDoneSuccess would have triggered once, on January 26 if set to 2h, instead of 4h.  A customer was potentially struggling with the outer loop. We hit a 60% error rate in Pyrra briefly: https://pyrra.gitpod.io/objectives?expr={__name__=%22workspace-imagebuild-buildsdone-success-ratio%22,%20namespace=%22monitoring-central%22,%20team=%22workspace%22}&grouping={}&from=1673297716785&to=1675716916785

GitpodImagebuildStartSuccess would have fired once, on Jan 8, when GCP was having scaling issues, and would have been correct to do so. https://gitpod.slack.com/archives/C01TNS8EVQT/p1673173223060219

Removed the warnings because they're unnecessary. Why? Pyrra sends them now for SLOs to #team-workspace-alerts.

Author: kylos101

operations/observability/mixins/workspace/rules/central/workspaces.yaml

@@ -91,25 +91,26 @@ spec:
       expr: |
         sum(time() - kube_pod_deletion_timestamp{namespace="default", pod=~"^ws-.*", cluster!~"ephemeral.*"}) by (pod) > 24 * 60 * 60
 
-    - alert: GitpodImagebuildSuccessRate
+    - alert: GitpodImagebuildDoneSuccess
       labels:
-        severity: warning
+        severity: critical
         team: workspace
+      for: 4h
       annotations:
-        runbook_url: https://github.com/gitpod-io/runbooks/blob/main/runbooks/GitpodImagebuildSuccessRate.md
-        summary: imagebuild success rate is low in cluster {{ $labels.cluster }}.
-        description: imagebuild are failing at too high of a rate in cluster {{ $labels.cluster }}.
+        runbook_url: https://github.com/gitpod-io/runbooks/blob/main/runbooks/GitpodImagebuildDoneSuccess.md
+        summary: imagebuilds done are failing at a high rate in cluster {{ $labels.cluster }}.
+        description: imagebuilds`s are not reaching done at too high of a rate in cluster {{ $labels.cluster }}.
       expr: |
-        (1 - (sum(rate(gitpod_image_builder_builds_done_total{success="false", cluster!~"ephemeral.*"}[4h])) / sum(rate(gitpod_image_builder_builds_done_total{cluster!~"ephemeral.*"}[4h])))) < 0.75
+        (1 - (sum(rate(gitpod_image_builder_builds_done_total{success="false", cluster!~"ephemeral.*"}[4h])) / sum(rate(gitpod_image_builder_builds_done_total{cluster!~"ephemeral.*"}[4h])))) < 0.60
 
-    - alert: GitpodImagebuildSuccessFailing
+    - alert: GitpodImagebuildStartSuccess
       labels:
         severity: critical
         team: workspace
-      for: 3h
+      for: 2h
       annotations:
-        runbook_url: https://github.com/gitpod-io/runbooks/blob/main/runbooks/GitpodImagebuildSuccessRate.md
-        summary: imagebuild success rate is failing in cluster {{ $labels.cluster }}.
-        description: imagebuild are failing at too high of a rate in cluster {{ $labels.cluster }}.
+        runbook_url: https://github.com/gitpod-io/runbooks/blob/main/runbooks/GitpodImagebuildStartSuccess.md
+        summary: imagebuild start success rate is failing in cluster {{ $labels.cluster }}.
+        description: imagebuild starts are failing at too high of a rate in cluster {{ $labels.cluster }}.
       expr: |
-        (1 - (sum(rate(gitpod_image_builder_builds_done_total{success="false", cluster!~"ephemeral.*"}[4h])) / sum(rate(gitpod_image_builder_builds_done_total{cluster!~"ephemeral.*"}[4h])))) < 0.50
+        (1 - (sum(rate(gitpod_ws_manager_workspace_starts_failure_total{type="IMAGEBUILD",cluster!~"ephemeral.*"}[4h])) / sum(rate(gitpod_ws_manager_workspace_starts_total{type="IMAGEBUILD", cluster!~"ephemeral.*"}[4h])))) < 0.99