-
-
Notifications
You must be signed in to change notification settings - Fork 461
Create Test Matrix for Spring Boot #4741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| c8125f3 | 397.65 ms | 485.14 ms | 87.49 ms |
| ee747ae | 386.94 ms | 431.43 ms | 44.49 ms |
| 23d6b12 | 354.10 ms | 408.38 ms | 54.28 ms |
| d217708 | 409.83 ms | 474.72 ms | 64.89 ms |
| d217708 | 411.22 ms | 430.86 ms | 19.63 ms |
| ee747ae | 400.46 ms | 423.61 ms | 23.15 ms |
| 17a0955 | 372.53 ms | 446.70 ms | 74.17 ms |
| 3d205d0 | 352.15 ms | 432.53 ms | 80.38 ms |
| 1df7eb6 | 397.04 ms | 429.64 ms | 32.60 ms |
| b750b96 | 408.98 ms | 480.32 ms | 71.34 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| c8125f3 | 1.58 MiB | 2.10 MiB | 532.32 KiB |
| ee747ae | 1.58 MiB | 2.10 MiB | 530.95 KiB |
| 23d6b12 | 1.58 MiB | 2.10 MiB | 532.31 KiB |
| d217708 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| d217708 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| ee747ae | 1.58 MiB | 2.10 MiB | 530.95 KiB |
| 17a0955 | 1.58 MiB | 2.10 MiB | 533.20 KiB |
| 3d205d0 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| 1df7eb6 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| b750b96 | 1.58 MiB | 2.10 MiB | 533.19 KiB |
Previous results on branch: feat/spring-boot-test-matrix
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 9594bb8 | 389.98 ms | 451.82 ms | 61.84 ms |
| 9d96865 | 372.85 ms | 440.84 ms | 67.99 ms |
| 70cd2c4 | 406.22 ms | 415.22 ms | 9.00 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 9594bb8 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
| 9d96865 | 1.58 MiB | 2.10 MiB | 532.96 KiB |
| 70cd2c4 | 1.58 MiB | 2.10 MiB | 532.97 KiB |
|
|
||
| - name: Exclude android modules from build | ||
| run: | | ||
| sed -i \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would it be easier and cleaner to create a script in the repo call it from the workflows?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes there's a lot here to clean up eventually. I still think it's already valuable enough to add and improve later.
| - name: Test sentry-samples-spring-boot | ||
| run: | | ||
| python3 test/system-test-runner.py test \ | ||
| --module "sentry-samples-spring-boot" \ | ||
| --agent false \ | ||
| --auto-init "true" \ | ||
| --build "true" | ||
|
|
||
| - name: Test sentry-samples-spring-boot-webflux | ||
| run: | | ||
| python3 test/system-test-runner.py test \ | ||
| --module "sentry-samples-spring-boot-webflux" \ | ||
| --agent false \ | ||
| --auto-init "true" \ | ||
| --build "true" | ||
|
|
||
| - name: Test sentry-samples-spring-boot-opentelemetry | ||
| run: | | ||
| python3 test/system-test-runner.py test \ | ||
| --module "sentry-samples-spring-boot-opentelemetry" \ | ||
| --agent true \ | ||
| --auto-init "true" \ | ||
| --build "true" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we have an additional matrix on top of the current one that governs agent and auto-init, so that we run all of these in parallel?
Or would that be overkill?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running the tests should just add a couple seconds to the job. I haven't figured out a good way to cache so I think we don't gain much by adding another matrix.
I'll add the auto-init false as another step here for now and we can revisit once we figure out caching / speed up the jobs.
#skip-changelog
📜 Description
💡 Motivation and Context
💚 How did you test it?
📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps