-
Notifications
You must be signed in to change notification settings - Fork 6k
iOS: Reduce rerun flakes by eliminating timeouts #55994
Conversation
When run repeatedly, some of the iOS unit tests can relatively easily blow their test timeouts. Since we're testing behaviour and not performance in these tests there's no need to enforce a particular timeout. The bots will fail if the tests timeout, regardless. Issue: flutter/flutter#157231
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm for removing the timeouts to reduce flakes, but I'd prefer if we keep the waitForExpectationsWithTimeout behavior that doesn't require each expectation to be explicitly awaited. It's too easy to miss new ones.
Can we make the timeout ridiculously large instead?
Same comment for #55961
|
Yeah we could define a shared constant somewhere. |
I'm surprised that Apple doesn't provide a version of API that doesn't take the timeout. Ya I agree that its very easy to forget some expectations this way. |
|
I was hoping that Apple would have provided a way to look up all created expectations but it seems like they also don't provide that -- if they did, we could just gather up the expectations from wherever apple is gathering them up, then invoke the method I used. Based on the documentation, the I actually personally prefer being explicit about what expectations we're creating and what expectations we're waiting for -- as a bonus, you can interleave creating and waiting for expectations -- but I don't feel super strongly about it, and if the consensus is we prefer to wait for everything, I'm more than happy to switch it back to that with a long timeout declared in a shared header. |
|
Can we land this now? Seems good to go. |
|
Adding a WIP tag so triagers don't trip on this. Please remove once progress is made. |
|
Thanks -- I'll get time to take a second pass at this this week. |
|
Since the engine repo is no longer a thing, I'm closing this. Plus I'm going to take a different approach with this. |
When run repeatedly, some of the iOS unit tests can relatively easily
blow their test timeouts. Since we're testing behaviour and not
performance in these tests there's no need to enforce a particular
timeout.
The bots will fail if the tests timeout, regardless.
Issue: flutter/flutter#157231
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.