Skip to content

Conversation

@firewave
Copy link
Collaborator

This script runs each test case separately so we can uncover cases where it depends on left-over data on the previous one. CTest is only running the fixtures separately so that does not.

@firewave
Copy link
Collaborator Author

My first attempt was to use a grep script to get all the names but because some test files contain multiple fixtures that does not work easily. So I went with any easier way and just get the file of all the tests from the application itself.

We currently cannot add this into the CI since the start-up time of testrunner is too high since it always instantiates all the test fixtures. Even with a release build the run-time is substantially higher. This issue is being tracked in https://trac.cppcheck.net/ticket/12080.

We can probably also integrate this into CTest but I did not look into it (will file a ticket about it though). But that also requires the run-time issue being resolved first.

@firewave firewave changed the title testrunner: add options -n (no dummary) and -d (dry run) / added script to run test case separately testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately Feb 27, 2024
@firewave
Copy link
Collaborator Author

"no dummary" 😵‍💫

@firewave firewave force-pushed the testrunner-single-2 branch from 2889549 to 06a1340 Compare February 27, 2024 22:51
@firewave firewave changed the title testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately / adjusted some tests Feb 28, 2024
@firewave firewave force-pushed the testrunner-single-2 branch from 06a1340 to ffbce1a Compare February 28, 2024 00:30
@firewave firewave marked this pull request as ready for review February 28, 2024 00:30
@firewave firewave marked this pull request as draft February 28, 2024 00:31
@firewave
Copy link
Collaborator Author

Done - but script still lacks the executable flag.

@firewave firewave force-pushed the testrunner-single-2 branch from ffbce1a to ea10f3a Compare February 28, 2024 12:10
@firewave firewave marked this pull request as ready for review February 28, 2024 12:13
@chrchr-github
Copy link
Collaborator

"no dummary" 😵‍💫

image

@firewave firewave force-pushed the testrunner-single-2 branch from ea10f3a to fd7a753 Compare February 28, 2024 13:28
@firewave firewave changed the title testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately / adjusted some tests testrunner: added options -n (no summary) and -d (dry run) / added script to run test cases separately / adjusted some tests Mar 4, 2024
@firewave firewave force-pushed the testrunner-single-2 branch from fd7a753 to 4cc8d84 Compare March 4, 2024 15:15
@firewave firewave force-pushed the testrunner-single-2 branch from 4cc8d84 to d3d562e Compare March 5, 2024 15:28
@firewave
Copy link
Collaborator Author

firewave commented Mar 6, 2024

Beside the dummary almost a whole week without any feedback - merging.

@firewave firewave merged commit 54c01cf into danmar:main Mar 6, 2024
@firewave firewave deleted the testrunner-single-2 branch March 6, 2024 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants