Skip to content

Commit 4ac0a5b

Browse files
committed
Use per exercise process timeout
1 parent f2b7401 commit 4ac0a5b

File tree

1 file changed

+5
-11
lines changed

1 file changed

+5
-11
lines changed

bin/test.sh

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -58,23 +58,17 @@ function test {
5858
cp "${exercise_dir}/.meta/${example_file}" "${outdir}/${exercise_file}.${file_ext}"
5959
fi
6060

61-
# The time-limit `18` is based on an approximation of the performance.
61+
# The time-limit `54` is based on an approximation of the performance.
6262
# Online Test Runner is appr. 3x faster than GitHub CI on Ubuntu.
6363
#
6464
# The total runtime limit of the Online Test Runner is 20 seconds including
6565
# Docker container launch and processing the results. That leaves appr. 18
66-
# seconds for all tests of an exercise. But that's not the `18` here.
66+
# seconds for all tests of an exercise.
6767
#
68-
# The longest running exercise tests in CI are in `alphametics`. It has 3 slow
69-
# test cases (9 letters and 10 letters) that require appr. 15 secs each in CI
70-
# and appr. 5 seconds each in the Online Test Runner.
71-
#
72-
# Putting that together gives a max. of 6 seconds for each of the slow test
73-
# cases in the Online Test Runner before hitting the timeout in production
74-
# -> use 3x 6secs = 18 seconds as a CI limit to ensure tests can actually
75-
# be run in production.
68+
# Putting that together gives a max. of 18 seconds x3 = 54 seconds for any
69+
# test class in CI.
7670
# See: https://forum.exercism.org/t/test-tracks-for-the-20-seconds-limit-on-test-runners/10536/8
77-
eval "${PHPUNIT_BIN}" --default-time-limit 18 --enforce-time-limit --fail-on-risky --no-configuration "${outdir}/${test_file}"
71+
timeout 54s "${PHPUNIT_BIN}" --no-configuration "${outdir}/${test_file}"
7872
}
7973

8074
function installed {

0 commit comments

Comments
 (0)