Skip to content

Conversation

@isabella-janssen
Copy link
Member

@isabella-janssen isabella-janssen commented Apr 11, 2025

Closes: OCPBUGS-54592

- What I did
This refactors some code originally written in #4982 to address OCPBUGS-54872. Due to the overlap of OCPBUGS-54592 (this PR's intended target) and OCPBUGS-54872 (fixed previously), no other code changes were required.

- How to verify it

  1. Apply a valid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application is successful, no LastFailedGeneration or LastFailedGenerationError value should be populated.
Example Valid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
  name: test-pinned
  labels:
    machineconfiguration.openshift.io/role: "worker"
spec:
  pinnedImages:
   - name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
   - name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
  Pinned Image Sets:
    Current Generation:  1
    Desired Generation:  1
    Name:                test-pinned
....
  1. Apply an invalid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application will fail, both the LastFailedGeneration and LastFailedGenerationError value should be populated.
Example Invalid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
  name: test-pinned
  labels:
    machineconfiguration.openshift.io/role: "worker"
spec:
  pinnedImages:
   - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079
   - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
  Pinned Image Sets:
    Desired Generation:            2
    Last Failed Generation:        2
    Last Failed Generation Error:  failed to execute podman manifest inspect for "quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079": exit status 125
    Name:                          test-pinned
....

- Description for the changelog
OCPBUGS-54592: Only populate LastFailedGeneration & LastFailedGenerationError values in MCN on PIS failure

@openshift-ci-robot openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 11, 2025
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-54592, which is invalid:

  • expected the bug to target the "4.19.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

TODO: Rebase once #4982 lands.

Closes: OCPBUGS-54592

- What I did
This refactors some code originally written in #4982 to address OCPBUGS-54872. Due to the overlap of OCPBUGS-54592 (this PR's intended target) and OCPBUGS-54872 (fixed previously), no other code changes were required.

- How to verify it

  1. Make sure PIS status is being reported correctly for a valid PIS application.
    a. Note that LastFailedGeneration should not report a value of 0.
Example Valid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
  - name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Current Generation:  1
   Desired Generation:  1
   Name:                test-pinned
....
  1. Make sure PIS status is being reported correctly for an invalid PIS application.
    a. Note that LastFailedGeneration should reflect that the PIS application has failed and LastFailedGenerationError should be populated.
Example PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Desired Generation:            2
   Last Failed Generation:        2
   Last Failed Generation Error:  failed to execute podman manifest inspect for "quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079": exit status 125
   Name:                          test-pinned
....

- Description for the changelog

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 11, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 11, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 11, 2025
@isabella-janssen
Copy link
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 11, 2025
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-54592, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.19.0) matches configured target version for branch (4.19.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from sergiordlr April 11, 2025 14:46
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-54592, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.19.0) matches configured target version for branch (4.19.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

Closes: OCPBUGS-54592

- What I did
This refactors some code originally written in #4982 to address OCPBUGS-54872. Due to the overlap of OCPBUGS-54592 (this PR's intended target) and OCPBUGS-54872 (fixed previously), no other code changes were required.

- How to verify it

  1. Apply a valid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application is successful, no LastFailedGeneration or LastFailedGenerationError value should be populated.
Example Valid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
  - name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Current Generation:  1
   Desired Generation:  1
   Name:                test-pinned
....
  1. Apply an invalid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application will fail, both the LastFailedGeneration and LastFailedGenerationError value should be populated.
Example Invalid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Desired Generation:            2
   Last Failed Generation:        2
   Last Failed Generation Error:  failed to execute podman manifest inspect for "quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079": exit status 125
   Name:                          test-pinned
....

- Description for the changelog
OCPBUGS-54592: Only populate LastFailedGeneration & LastFailedGenerationError values in MCN on PIS failure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@isabella-janssen isabella-janssen marked this pull request as ready for review April 11, 2025 18:50
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 11, 2025
@umohnani8
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 17, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 17, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: isabella-janssen, umohnani8

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [isabella-janssen,umohnani8]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@isabella-janssen
Copy link
Member Author

/hold

Holding for QE.

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 17, 2025
@ptalgulk01
Copy link

ptalgulk01 commented Apr 21, 2025

Pre-merge verified:
Steps are similar to here
Verified using IPI based GCP TechPreview enable 4.19 cluster.

  1. Before applying any MC/PIS following condition was seen in MCN for master & worker pool
$ oc describe machineconfignode ci-ln-wkzm86b-72292-8fnpn-worker-c-z9w8n | tail -n 10
    Last Transition Time:  2025-04-21T06:58:38Z
    Message:               All is good
    Reason:                AsExpected
    Status:                False
    Type:                  PinnedImageSetsDegraded
  Config Version:
    Current:            rendered-worker-56e0a8f5585ffca778ea327471a3fb69
    Desired:            rendered-worker-56e0a8f5585ffca778ea327471a3fb69
  Observed Generation:  4
Events:                 
  1. Applied the MC on master & worker pool. During the MCN update the ObservedGeneration count was increased
$ oc describe machineconfignode ci-ln-wkzm86b-72292-8fnpn-worker-c-z9w8n | tail -n 10
    Last Transition Time:  2025-04-21T06:58:38Z
    Message:               All is good
    Reason:                AsExpected
    Status:                False
    Type:                  PinnedImageSetsDegraded
  Config Version:
    Current:            rendered-worker-56e0a8f5585ffca778ea327471a3fb69
    Desired:            rendered-worker-f4bbd63607a0a44cbdf3282054f14d5f
  Observed Generation:  5
Events:                 
  1. Applied in-valid PIS
Example
oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: pinned-bad-image-tc-73361
spec:
  pinnedImages:
  - name: "quay.io/openshiftfake/fakeimage@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019"
EOF
pinnedimageset.machineconfiguration.openshift.io/pinned-bad-image-tc-73361 created
- In MCN for in-valid PIS, both the last failed generation and last failed generation error should show in Pinned Image Sets field.
$ oc describe machineconfignode ci-ln-wkzm86b-72292-8fnpn-worker-c-z9w8n | tail -n 10
  Config Version:
    Current:            rendered-worker-f4bbd63607a0a44cbdf3282054f14d5f
    Desired:            rendered-worker-f4bbd63607a0a44cbdf3282054f14d5f
  Observed Generation:  5
  Pinned Image Sets:
    Desired Generation:            1
    Last Failed Generation:        1
    Last Failed Generation Error:  failed to execute podman manifest inspect for "quay.io/openshiftfake/fakeimage@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019": exit status 125
    Name:                          pinned-bad-image-tc-73361
Events:                            
4. Valid PIS
Example
oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: tc-80334-worker-pinned-images
spec:
  pinnedImages:
  - name: "quay.io/openshifttest/busybox@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019"
  - name: quay.io/openshifttest/alpine@sha256:be92b18a369e989a6e86ac840b7f23ce0052467de551b064796d67280dfa06d5
EOF
pinnedimageset.machineconfiguration.openshift.io/tc-80334-worker-pinned-images created
- In MCN following was observed. Verifying the no last failed generation or last failed generation error should show in the MCN
$ oc describe machineconfignode ci-ln-wkzm86b-72292-8fnpn-worker-c-z9w8n | tail -n 10
  Config Version:
    Current:            rendered-worker-f4bbd63607a0a44cbdf3282054f14d5f
    Desired:            rendered-worker-f4bbd63607a0a44cbdf3282054f14d5f
  Observed Generation:  5
  Pinned Image Sets:
    Current Generation:  1
    Desired Generation:  1
    Name:                tc-80334-worker-pinned-images
Events:                  

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Apr 21, 2025
@isabella-janssen
Copy link
Member Author

/unhold

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 21, 2025
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 82ece18 and 2 for PR HEAD 8262eda in total

@isabella-janssen
Copy link
Member Author

/retest-required

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 01141a2 and 1 for PR HEAD 8262eda in total

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 01141a2 and 2 for PR HEAD 8262eda in total

@isabella-janssen
Copy link
Member Author

/retest-required

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 22, 2025

@isabella-janssen: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/bootstrap-unit 8262eda link false /test bootstrap-unit
ci/prow/e2e-azure-ovn-upgrade-out-of-change 8262eda link false /test e2e-azure-ovn-upgrade-out-of-change
ci/prow/okd-scos-e2e-aws-ovn 8262eda link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 01141a2 and 2 for PR HEAD 8262eda in total

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 574295e and 1 for PR HEAD 8262eda in total

@openshift-merge-bot openshift-merge-bot bot merged commit e9c0b18 into openshift:main Apr 23, 2025
14 of 17 checks passed
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: Jira Issue OCPBUGS-54592: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-54592 has been moved to the MODIFIED state.

In response to this:

Closes: OCPBUGS-54592

- What I did
This refactors some code originally written in #4982 to address OCPBUGS-54872. Due to the overlap of OCPBUGS-54592 (this PR's intended target) and OCPBUGS-54872 (fixed previously), no other code changes were required.

- How to verify it

  1. Apply a valid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application is successful, no LastFailedGeneration or LastFailedGenerationError value should be populated.
Example Valid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
  - name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Current Generation:  1
   Desired Generation:  1
   Name:                test-pinned
....
  1. Apply an invalid PIS and make sure the PIS ref in the MCN is reporting correctly. Note that since the application will fail, both the LastFailedGeneration and LastFailedGenerationError value should be populated.
Example Invalid PIS
apiVersion: machineconfiguration.openshift.io/v1
kind: PinnedImageSet
metadata:
 name: test-pinned
 labels:
   machineconfiguration.openshift.io/role: "worker"
spec:
 pinnedImages:
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079
  - name: quay.io/rh-ee-ijanssen/machine-config-operator@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b
$ oc apply -f <pis-file>
$ oc describe machineconfignode/<node-name>
...
Status:
...
 Pinned Image Sets:
   Desired Generation:            2
   Last Failed Generation:        2
   Last Failed Generation Error:  failed to execute podman manifest inspect for "quay.io/rh-ee-ijanssen/machine-config-operator@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079": exit status 125
   Name:                          test-pinned
....

- Description for the changelog
OCPBUGS-54592: Only populate LastFailedGeneration & LastFailedGenerationError values in MCN on PIS failure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

Distgit: ose-machine-config-operator
This PR has been included in build ose-machine-config-operator-container-v4.20.0-202504230513.p0.ge9c0b18.assembly.stream.el9.
All builds following this will include this PR.

@isabella-janssen isabella-janssen deleted the ocpbugs-54592 branch April 24, 2025 18:39
@isabella-janssen
Copy link
Member Author

/cherry-pick release-4.18

@openshift-cherrypick-robot

@isabella-janssen: #4983 failed to apply on top of branch "release-4.18":

Applying: upgrade monitor: only set LastFailedGeneration and LastFailedGenerationError on PIS failure
Using index info to reconstruct a base tree...
M	pkg/upgrademonitor/upgrade_monitor.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/upgrademonitor/upgrade_monitor.go
CONFLICT (content): Merge conflict in pkg/upgrademonitor/upgrade_monitor.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 upgrade monitor: only set LastFailedGeneration and LastFailedGenerationError on PIS failure

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants