Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@nturgut
Copy link
Contributor

@nturgut nturgut commented Aug 10, 2020

build_filter functionality was giving an error for windows for while therefore in windows tests were not building.

This PR changes test_runner.dart to build unit tests with dart2js instead of build_runner.

It uses pool to run builds in parallel. Time for tests for 8 core Macbook:

  • Prior to this PR tests were building in total 224 seconds.
  • When the new approach used with only thread it builds in 586 seconds.
  • Used a pool size with 8 to parallel the builds, it takes 173 seconds.

Fixes: flutter/flutter#62361 After this fix the Windows developers will be able to run felt_windows.bat test again. In order to make this step run also adding other small changes to the felt tool.

@nturgut nturgut requested a review from yjbanov August 10, 2020 22:26
@nturgut nturgut force-pushed the unit_tests_redesign branch from 4781ac0 to a124ec8 Compare August 12, 2020 00:23
@nturgut nturgut force-pushed the unit_tests_redesign branch from 8693597 to 4fad778 Compare August 13, 2020 00:39
TestTypesRequested testTypesRequested = null;

/// How many dart2js build tasks are running at the same time.
final Pool _pool = Pool(8);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use BUILD_MAX_WORKERS_PER_TASK as the pool size. Also, currently we default to 4. I recall you and @mdebbar preferring 4 as the default back when I introduced BUILD_MAX_WORKERS_PER_TASK.

Copy link
Contributor Author

@nturgut nturgut Aug 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"We should use BUILD_MAX_WORKERS_PER_TASK as the pool size" -> Why? Can you add a link if you made a prior analysis.

Also Why @mdebbar prefers 4?

The highest priority now for Flutter team is to make tree green. One thing we should do is to drop run time for LUCI tasks. I confirmed that all LUCI bots have at least 8 cores. I am ok with an argument for making this higher, I don't see it is 4?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it was just a conversation over VC. The only documented part was this: #17616 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"We should use BUILD_MAX_WORKERS_PER_TASK as the pool size" -> Why? Can you add a link if you made a prior analysis.

Because I am using 32 as the pool size, and it speed up my builds considerably. I added this capability in #17616. Currently, this change removes the capability to configure the pool size.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, the argument was not every machine has 16 cores. Right now all LUCI machines have at least 8 cores so we can use 8.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the argument for 4 cores was in favor of contributors using laptops. A lot of laptops have only 4 cores (although typically they are hyper-threaded, so 8 might still be OK).

@nturgut nturgut requested a review from yjbanov August 15, 2020 02:15
TestTypesRequested testTypesRequested = null;

/// How many dart2js build tasks are running at the same time.
final Pool _pool = Pool(8);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it was just a conversation over VC. The only documented part was this: #17616 (comment)

}

// Currently iOS Safari tests are running on simulator, which does not
// support canvaskit backend.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to put this comment here? I'm having trouble connecting it to the if block below. There's nothing in the code related to iOS, Safari, or simulators.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if anyone is reading the code, hoping to understand how a test runs, I'm telling them "be aware, these lines won't run on iOS Safari"

This is also in the skips in the individual tests but I think it's good to leave these stories to the readers.

@nturgut nturgut requested a review from yjbanov August 17, 2020 21:36
@yjbanov
Copy link
Contributor

yjbanov commented Aug 18, 2020

After this change my local builds sped up drastically from 2.5 minutes down to 49 seconds!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[web] Windows Web engine build is failing.

3 participants