Skip to content

Conversation

@sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Dec 10, 2018

…able

List of slow tests based on Jacob's list. I only included tests
that take more than 20 seconds.

Future work:

  • Warn if test marked as slow runs in <10 seconds.
  • Warn if test not marked as slow runs in >30 seconds.

…able

List of slow tests based on Jabob's list.  I only included tests
that take more than 20 seconds.

Future work:
- Warn if test marked as slow runs in <10 seconds.
- Warn if test not marked as slow runs in >30 seconds.
Copy link
Contributor

@jgravelle-google jgravelle-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/IWTEST_/EMTEST_/ in the commit message

lgtm

@sbc100 sbc100 changed the title Add is_slow_test test decorator and IWTEST_SKIP_SLOW environment vari… Add is_slow_test test decorator and EMTEST_SKIP_SLOW environment vari… Dec 10, 2018
@sbc100 sbc100 merged commit 9ecca00 into incoming Dec 11, 2018
@sbc100 sbc100 deleted the is_slow_test branch December 11, 2018 01:24
@kripken
Copy link
Member

kripken commented Dec 11, 2018

Just curious, what's the motivation here?

@jgravelle-google
Copy link
Contributor

Two parts to it:

  1. Some of these tests are very slow. test_freetype takes ~2 minutes on my machine. Running 500 other not-slow tests takes ~2 minutes on my machine too. (this is with the wasm backend) So for local testing it can be easier to sanity-check with a faster subset of tests, and only run the full testsuite for submit-ready PRs.
  2. For test scheduling, having a list of slow tests should let us start the longer tasks first more precisely than reverse-alphabetically sorting them, which should slightly speed up the full-testsuite case as well.

TODO: we should rename test_the_bullet to just test_bullet when we use this to affect test scheduling.

@kripken
Copy link
Member

kripken commented Dec 11, 2018

Yeah, we should sort the names in an optimal way.

If you just want a quick sanity check, you can do ./tests/runner.py random50 (or some other number). Or hand-pick e.g. binaryen2.test_c*.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants