Skip to content

NO-JIRA: Clarify test names in OTE #1256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

NO-JIRA: Clarify test names in OTE #1256

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion cmd/cluster-version-operator-tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,40 @@ After [installing-ginkgo](https://onsi.github.io/ginkgo/#installing-ginkgo):
$ ginkgo ./test/...
```

The output looks nicer this way.
The output looks nicer this way.


## Test Names

### Test labels

See [docs](https://github.com/openshift/origin/tree/main/test/extended#test-labels) for details.

### Ownership

* A `[Jira:"Component"]` tag in the test name (e.g., [Jira:"Cluster Version Operator"]) is preferred to claim the ownership of the test. The component comes from [the list of components of Jira Project OCPBUGS](https://issues.redhat.com/projects/OCPBUGS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page). See [here](https://github.com/openshift/origin/blob/6b584254d53cdd1b5cd6471b69cb7c22f3e28ecd/test/extended/apiserver/rollout.go#L36) for example.

* Or an entry in the [ci-test-mapping](https://github.com/openshift-eng/ci-test-mapping) repository, which maps tests to owning components. See [here](https://github.com/openshift-eng/ci-test-mapping/tree/main/pkg/components/clusterversionoperator) for example.

### Renaming

If a test is renamed, claim it with [specs.Walk](https://pkg.go.dev/github.com/openshift-eng/[email protected]/pkg/extension/extensiontests#ExtensionTestSpecs.Walk):
Copy link
Contributor

@DavidHurta DavidHurta Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clicking the link results in:

Image


```go
specs = specs.Walk(func(spec *et.ExtensionTestSpec) {
if orig, ok := renamedTests[spec.Name]; ok {
spec.OriginalName = orig
}
})
```

### Deleting

If a test is renamed, claim it with [ext.IgnoreObsoleteTests](https://pkg.go.dev/github.com/openshift-eng/[email protected]/pkg/extension#Extension.IgnoreObsoleteTests)
Copy link
Contributor

@DavidHurta DavidHurta Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If a test is renamed, claim it with [ext.IgnoreObsoleteTests](https://pkg.go.dev/github.com/openshift-eng/[email protected]/pkg/extension#Extension.IgnoreObsoleteTests)
If a test is deleted, claim it with [ext.IgnoreObsoleteTests](https://pkg.go.dev/github.com/openshift-eng/[email protected]/pkg/extension#Extension.IgnoreObsoleteTests)

Copy link
Contributor

@DavidHurta DavidHurta Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clicking the link results in:

Image


```go
ext.IgnoreObsoleteTests(
"[sig-abc] My removed test",
// Add more removed test names below
)
```
6 changes: 5 additions & 1 deletion hack/update-test-metadata.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,8 @@ set -eu
source hack/build-info.sh

# Update test metadata
eval "${BIN_PATH}/cluster-version-operator-tests" "update"
if ! "${BIN_PATH}/cluster-version-operator-tests" "update"; then
Copy link
Contributor

@DavidHurta DavidHurta Nov 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment not related to the code

We can break down the commit into two distinct commits as the current commit updates the README and also de facto updates the default logic of the make update target, and this way also updates the logic of the verify-update job, which are quite distinct areas. What do you think?

>&2 echo "run 'make update' to fix the metadata after going through the teest renaming process.
See https://github.com/openshift/cluster-version-operator/blob/main/cmd/cluster-version-operator-tests/README.md#test-names for details."
Comment on lines +9 to +10
Copy link
Contributor

@DavidHurta DavidHurta Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the OTE enhancement update section:

If an incompatible change is introduced from the prior invocation of update (e.g. changing a test name without preserving the original), update will raise an error which the component owner must correct before committing their change in git.

Thus, it may not be just a renaming issue. Personally, I would make the message more general in nature; however, we can still reference the README file that can contain additional help.

exit 1
fi