- 
                Notifications
    You must be signed in to change notification settings 
- Fork 159
Speed up directory monitoring tests #1325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| @swift-ci please test | 
The directory monitoring tests check for whether the documentation is rebuilt if the documentation catalog is edited when running a live server with `docc preview`. These tests assumed an extremely generous timeout for receiving the directory change signal, and ran much slower than the rest of the test suite. The timeouts have been reduced to a more reasonable threshold that succeeds even with throttled resources on a single-core machine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only one of these changes make the directory monitor test faster, the rest only make them more fragile without running faster.
| wait(for: [updated], timeout: 5.0) | ||
| wait(for: [updated], timeout: 1.5) | ||
| } | ||
|  | ||
| if isTreeReloadExpected { | ||
| wait(for: [reloaded], timeout: 5.0) | ||
| wait(for: [reloaded], timeout: 1.5) | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These timeouts don't speed up the test as long as it passes. Even if I increase both of these to 100 second testMonitorUpdates only takes ~5 seconds for me to run locally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we want this to succeed faster then we'd need to change the 1 second throttle time here.
| } | ||
|  | ||
| wait(for: [didNotTriggerUpdateForHiddenFile], timeout: 20) | ||
| wait(for: [didNotTriggerUpdateForHiddenFile], timeout: 2) | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also doesn't speed up the test as long as it passes. Even if I increase both of these to 200 second testMonitorDoesNotTriggerUpdates only takes ~20 seconds for me to run locally.
Summary
The directory monitoring tests check for whether the documentation is rebuilt if the documentation catalog is edited when running a live server with
docc preview. These tests assumed an extremely generous timeout of 5 seconds for changes to the directory, and 10 seconds for a lack of change. Thus, they ran much slower than the rest of the test suite. The timeouts have been reduced to a more reasonable threshold of 1.5 seconds, which still succeeds with throttled CPU resources.Dependencies
N/A
Testing
This was tested by restricting CPU resources to 20% of a single core and running the tests 100 times:
Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.
[ ] Added tests./bin/testscript and it succeeded[ ] Updated documentation if necessary