diff --git a/documentation/src/docs/asciidoc/release-notes/release-notes-5.5.0-RC1.adoc b/documentation/src/docs/asciidoc/release-notes/release-notes-5.5.0-RC1.adoc index 37b159a51346..1c7cced0275b 100644 --- a/documentation/src/docs/asciidoc/release-notes/release-notes-5.5.0-RC1.adoc +++ b/documentation/src/docs/asciidoc/release-notes/release-notes-5.5.0-RC1.adoc @@ -55,6 +55,8 @@ on GitHub. ==== New Features and Improvements +* Support for declarative timeouts using `@Timeout` or configuration parameters (see + <<../user-guide/index.adoc#writing-tests-declarative-timeouts, User Guide>> for details) * New overloaded variants of `Assertions.assertLinesMatch(...)` that accept a `String` or a `Supplier` for a custom failure message. * Failure messages for `Assertions.assertLinesMatch(...)` now emit each expected and diff --git a/documentation/src/docs/asciidoc/user-guide/writing-tests.adoc b/documentation/src/docs/asciidoc/user-guide/writing-tests.adoc index 374d87e0663c..7443878e7a8d 100644 --- a/documentation/src/docs/asciidoc/user-guide/writing-tests.adoc +++ b/documentation/src/docs/asciidoc/user-guide/writing-tests.adoc @@ -40,6 +40,7 @@ in the `junit-jupiter-api` module. | `@Nested` | Denotes that the annotated class is a non-static <>. `@BeforeAll` and `@AfterAll` methods cannot be used directly in a `@Nested` test class unless the "per-class" <> is used. Such annotations are not _inherited_. | `@Tag` | Used to declare <>, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4. Such annotations are _inherited_ at the class level but not at the method level. | `@Disabled` | Used to <> a test class or test method; analogous to JUnit 4's `@Ignore`. Such annotations are not _inherited_. +| `@Timeout` | Used to fail a test, test factory, test template, or lifecycle method if its execution exceeds a given duration. Such annotations are _inherited_. | `@ExtendWith` | Used to <>. Such annotations are _inherited_. | `@RegisterExtension` | Used to <> via fields. Such fields are _inherited_ unless they are _shadowed_. | `@TempDir` | Used to supply a <> via field injection or parameter injection in a lifecycle method or test method; located in the `org.junit.jupiter.api.io` package. @@ -217,11 +218,12 @@ include::{testDir}/example/AssertionsDemo.java[tags=user_guide] [[writing-tests-assertions-preemptive-timeouts]] [WARNING] -.Preemptive Timeouts +.Preemptive Timeouts with `assertTimeoutPreemptively()` ==== -The various `assertTimeoutPreemptively()` methods in the `Assertions` class execute the -provided `executable` or `supplier` in a different thread than that of the calling code. -This behavior can lead to undesirable side effects if the code that is executed within the +Contrary to <>, the various +`assertTimeoutPreemptively()` methods in the `Assertions` class execute the provided +`executable` or `supplier` in a different thread than that of the calling code. This +behavior can lead to undesirable side effects if the code that is executed within the `executable` or `supplier` relies on `java.lang.ThreadLocal` storage. One common example of this is the transactional testing support in the Spring Framework. @@ -1569,6 +1571,95 @@ include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide] ---- +[[writing-tests-declarative-timeouts]] +=== Timeouts + +.Declarative timeouts are an experimental feature +WARNING: You're invited to give it a try and provide feedback to the JUnit team so they +can improve and eventually <> this feature. + +The `@Timeout` annotation allows to declare that a test, test factory, test template, or +lifecycle method should fail if its execution time exceeds a given duration, and +optionally a time unit (seconds are used by default). + +The following example shows how `@Timeout` is applied to lifecycle and test methods. + +[source,java] +---- +include::{testDir}/example/TimeoutDemo.java[tags=user_guide] +---- + +Contrary to the `assertTimeoutPreemptively()` assertion, the execution of the annotated +method proceeds in the main thread of the test. If the timeout is exceeded, the main +thread is interrupted from another thread. This is done to ensure interoperability with +frameworks such as Spring that make extensive use of mechanisms that are sensitive to the +currently running thread (e.g. `ThreadLocals`). + +To apply the same timeout to all test methods within a test class and all its `@Nested` +classes, you can declare the `@Timeout` annotation on the class level. It will then be +applied to all test, test factory, and test template methods within that class and its +`@Nested` classes unless overridden by a `@Timeout` annotation on a method or `@Nested` +class. Please note that `@Timeout` annotations declared on the class level are not +applied to lifecycle methods. + +Declaring `@Timeout` on a test factory method checks that the method returns within the +specified duration but does verify the execution time of the returned `DynamicTests`. +Please use `assertTimeout()`/`assertTimeoutPreemptively()` for that purpose. + +If `@Timeout` is present on a test template method, such as a method annotated with +`@RepeatedTest` or `@ParameterizedTest`, each invocation is checked to finish within the +given timeout. + +The following <> can be used to +specify global timeouts for all methods of a certain category unless they or an enclosing +test class is annotated with `@Timeout`: + +`junit.jupiter.execution.timeout.default`:: + Default timeout for all testable and lifecycle methods +`junit.jupiter.execution.timeout.testable.method.default`:: + Default timeout for all testable methods +`junit.jupiter.execution.timeout.test.method.default`:: + Default timeout for `@Test` methods +`junit.jupiter.execution.timeout.testtemplate.method.default`:: + Default timeout for `@TestTemplate` methods +`junit.jupiter.execution.timeout.testfactory.method.default`:: + Default timeout for `@TestFactory` methods +`junit.jupiter.execution.timeout.lifecycle.method.default`:: + Default timeout for all lifecycle methods +`junit.jupiter.execution.timeout.beforeall.method.default`:: + Default timeout for `@BeforeAll` methods +`junit.jupiter.execution.timeout.beforeeach.method.default`:: + Default timeout for `@BeforeEach` methods +`junit.jupiter.execution.timeout.aftereach.method.default`:: + Default timeout for `@AfterEach` methods +`junit.jupiter.execution.timeout.afterall.method.default`:: + Default timeout for `@AfterAll` methods + +More specific configuration parameters override less specific ones. For example, +`junit.jupiter.execution.timeout.test.method.default` overrides +`junit.jupiter.execution.timeout.testable.method.default` which overrides +`junit.jupiter.execution.timeout.default`. + +The values of such configuration parameters must be in the following, case-insensitive +format: ` [ns|μs|ms|s|m|h|d]`. The space between the number and the unit may be +omitted. Specifying no unit is equivalent to using seconds. + +.Example timeout configuration parameter values +[cols="20,80"] +|=== +| Parameter value | Equivalent annotation + +| `42` | `@Timeout(42)` +| `42 ns` | `@Timeout(value = 42, unit = NANOSECONDS)` +| `42 μs` | `@Timeout(value = 42, unit = MICROSECONDS)` +| `42 ms` | `@Timeout(value = 42, unit = MILLISECONDS)` +| `42 s` | `@Timeout(value = 42, unit = SECONDS)` +| `42 m` | `@Timeout(value = 42, unit = MINUTES)` +| `42 h` | `@Timeout(value = 42, unit = HOURS)` +| `42 d` | `@Timeout(value = 42, unit = DAYS)` +|=== + + [[writing-tests-parallel-execution]] === Parallel Execution diff --git a/documentation/src/test/java/example/TimeoutDemo.java b/documentation/src/test/java/example/TimeoutDemo.java new file mode 100644 index 000000000000..103dbba54d06 --- /dev/null +++ b/documentation/src/test/java/example/TimeoutDemo.java @@ -0,0 +1,35 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package example; + +import java.util.concurrent.TimeUnit; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.Timeout; + +// tag::user_guide[] +class TimeoutDemo { + + @BeforeEach + @Timeout(5) + void setUp() { + // fails if execution time exceeds 5 seconds + } + + @Test + @Timeout(value = 100, unit = TimeUnit.MILLISECONDS) + void failsIfExecutionTimeExceedsFiveSeconds() { + // fails if execution time exceeds 100 milliseconds + } + +} +// end::user_guide[] diff --git a/junit-jupiter-api/src/main/java/org/junit/jupiter/api/Timeout.java b/junit-jupiter-api/src/main/java/org/junit/jupiter/api/Timeout.java new file mode 100644 index 000000000000..3543ec7e7b59 --- /dev/null +++ b/junit-jupiter-api/src/main/java/org/junit/jupiter/api/Timeout.java @@ -0,0 +1,111 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.api; + +import static org.apiguardian.api.API.Status.EXPERIMENTAL; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Inherited; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.concurrent.TimeUnit; + +import org.apiguardian.api.API; + +/** + * {@code @Timeout} is used to define a timeout for a method or all testable + * methods within one class and its {@link Nested @Nested} classes. + * + *

This annotation may also be used on lifecycle methods annotated with + * {@link BeforeAll @BeforeAll}, {@link BeforeEach @BeforeEach}, + * {@link AfterEach @AfterEach}, or {@link AfterAll @AfterAll}. + * + *

Applying this annotation to a test class has the same effect as applying + * it to all testable methods, i.e. all methods annotated or meta-annotated with + * {@link Test @Test}, {@link TestFactory @TestFactory}, or + * {@link TestTemplate @TestTemplate}, but not to its lifecycle methods. + * + *

Default Timeouts

+ * + *

If this annotation is not present, no timeout will be used unless a + * default timeout is defined via one of the following configuration parameters: + * + *

+ *
{@code junit.jupiter.execution.timeout.default}
+ *
Default timeout for all testable and lifecycle methods
+ *
{@code junit.jupiter.execution.timeout.testable.method.default}
+ *
Default timeout for all testable methods
+ *
{@code junit.jupiter.execution.timeout.test.method.default}
+ *
Default timeout for {@link Test @Test} methods
+ *
{@code junit.jupiter.execution.timeout.testtemplate.method.default}
+ *
Default timeout for {@link TestTemplate @TestTemplate} methods
+ *
{@code junit.jupiter.execution.timeout.testfactory.method.default}
+ *
Default timeout for {@link TestFactory @TestFactory} methods
+ *
{@code junit.jupiter.execution.timeout.lifecycle.method.default}
+ *
Default timeout for all lifecycle methods
+ *
{@code junit.jupiter.execution.timeout.beforeall.method.default}
+ *
Default timeout for {@link BeforeAll @BeforeAll} methods
+ *
{@code junit.jupiter.execution.timeout.beforeeach.method.default}
+ *
Default timeout for {@link BeforeEach @BeforeEach} methods
+ *
{@code junit.jupiter.execution.timeout.aftereach.method.default}
+ *
Default timeout for {@link AfterEach @AfterEach} methods
+ *
{@code junit.jupiter.execution.timeout.afterall.method.default}
+ *
Default timeout for {@link AfterAll @AfterAll} methods
+ *
+ * + *

More specific configuration parameters override less specific ones. For + * example, {@code junit.jupiter.execution.timeout.test.method.default} + * overrides {@code junit.jupiter.execution.timeout.testable.method.default} + * which overrides {@code junit.jupiter.execution.timeout.default}. + * + *

Values must be in the following, case-insensitive format: + * {@code [ns|μs|ms|s|m|h|d]}. The space between the number and the + * unit may be omitted. Specifying no unit is equivalent to using seconds. + * + * + * + * + * + * + * + * + * + * + * + *
Value Equivalent annotation
{@code 42} {@code @Timeout(42)}
{@code 42 ns} {@code @Timeout(value = 42, unit = NANOSECONDS)}
{@code 42 μs} {@code @Timeout(value = 42, unit = MICROSECONDS)}
{@code 42 ms} {@code @Timeout(value = 42, unit = MILLISECONDS)}
{@code 42 s} {@code @Timeout(value = 42, unit = SECONDS)}
{@code 42 m} {@code @Timeout(value = 42, unit = MINUTES)}
{@code 42 h} {@code @Timeout(value = 42, unit = HOURS)}
{@code 42 d} {@code @Timeout(value = 42, unit = DAYS)}
+ * + * @since 5.5 + */ +@Target({ ElementType.TYPE, ElementType.METHOD }) +@Retention(RetentionPolicy.RUNTIME) +@Documented +@Inherited +@API(status = EXPERIMENTAL, since = "5.5") +public @interface Timeout { + + /** + * The duration of this timeout. + * + * @return timeout duration; must be a positive number + */ + long value(); + + /** + * The time unit of this timeout. + * + * @return time unit + * @see TimeUnit + */ + TimeUnit unit() default TimeUnit.SECONDS; + +} diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java index f2af71cb51fe..fc35e5514b5b 100644 --- a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java @@ -24,6 +24,24 @@ /** * Collection of constants related to the {@link JupiterTestEngine}. * + *

Supported Values for Timeouts

+ * + *

Values for timeouts must be in the following, case-insensitive format: + * {@code [ns|μs|ms|s|m|h|d]}. The space between the number and the + * unit may be omitted. Specifying no unit is equivalent to using seconds. + * + * + * + * + * + * + * + * + * + * + * + *
Value Equivalent annotation
{@code 42} {@code @Timeout(42)}
{@code 42 ns} {@code @Timeout(value = 42, unit = NANOSECONDS)}
{@code 42 μs} {@code @Timeout(value = 42, unit = MICROSECONDS)}
{@code 42 ms} {@code @Timeout(value = 42, unit = MILLISECONDS)}
{@code 42 s} {@code @Timeout(value = 42, unit = SECONDS)}
{@code 42 m} {@code @Timeout(value = 42, unit = MINUTES)}
{@code 42 h} {@code @Timeout(value = 42, unit = HOURS)}
{@code 42 d} {@code @Timeout(value = 42, unit = DAYS)}
+ * * @see org.junit.platform.engine.ConfigurationParameters * @since 5.0 */ @@ -187,6 +205,205 @@ public final class Constants { public static final String PARALLEL_CONFIG_CUSTOM_CLASS_PROPERTY_NAME = PARALLEL_CONFIG_PREFIX + CONFIG_CUSTOM_CLASS_PROPERTY_NAME; + /** + * Property name used to set the default timeout for all testable and + * lifecycle methods. + * + *

The value of this property will be used unless overridden by a more + * specific property or a {@link org.junit.jupiter.api.Timeout @Timeout} + * annotation present on the method or an enclosing test class (for testable + * methods). + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all testable methods. + * + *

The value of this property will be used unless overridden by a more + * specific property or a {@link org.junit.jupiter.api.Timeout @Timeout} + * annotation present on the testable method or an enclosing test class. + * + *

This property overrides the {@value #DEFAULT_TIMEOUT_PROPERTY_NAME} + * property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.Test @Test} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.Test @Test} method or an enclosing test + * class. + * + *

This property overrides the + * {@value #DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.TestTemplate @TestTemplate} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.TestTemplate @TestTemplate} method or an + * enclosing test class. + * + *

This property overrides the + * {@value #DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.TestFactory @TestFactory} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.TestFactory @TestFactory} method or an + * enclosing test class. + * + *

This property overrides the + * {@value #DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all lifecycle methods. + * + *

The value of this property will be used unless overridden by a more + * specific property or a {@link org.junit.jupiter.api.Timeout @Timeout} + * annotation present on the lifecycle method. + * + *

This property overrides the {@value #DEFAULT_TIMEOUT_PROPERTY_NAME} + * property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.BeforeAll @BeforeAll} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.BeforeAll @BeforeAll} method. + * + *

This property overrides the + * {@value #DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.BeforeEach @BeforeEach} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.BeforeEach @BeforeEach} method. + * + *

This property overrides the + * {@value #DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.AfterEach @AfterEach} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.AfterEach @AfterEach} method. + * + *

This property overrides the + * {@value #DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME; + + /** + * Property name used to set the default timeout for all + * {@link org.junit.jupiter.api.AfterAll @AfterAll} methods. + * + *

The value of this property will be used unless overridden by a + * {@link org.junit.jupiter.api.Timeout @Timeout} annotation present on the + * {@link org.junit.jupiter.api.AfterAll @AfterAll} method. + * + *

This property overrides the + * {@value #DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME} property. + * + *

Please refer to the class + * description for the definition of supported values. + * + * @see org.junit.jupiter.api.Timeout + * @since 5.5 + */ + @API(status = EXPERIMENTAL, since = "5.5") + public static final String DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME = JupiterConfiguration.DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME; + private Constants() { /* no-op */ } diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/JupiterConfiguration.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/JupiterConfiguration.java index 9326f9c11f01..474a1c4dd4db 100644 --- a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/JupiterConfiguration.java +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/JupiterConfiguration.java @@ -35,6 +35,17 @@ public interface JupiterConfiguration { String DEACTIVATE_ALL_CONDITIONS_PATTERN = ClassNamePatternParameterConverter.DEACTIVATE_ALL_PATTERN; String DEFAULT_DISPLAY_NAME_GENERATOR_PROPERTY_NAME = "junit.jupiter.displayname.generator.default"; + String DEFAULT_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.default"; + String DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.testable.method.default"; + String DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.test.method.default"; + String DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.testtemplate.method.default"; + String DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.testfactory.method.default"; + String DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.lifecycle.method.default"; + String DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.beforeall.method.default"; + String DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.beforeeach.method.default"; + String DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.aftereach.method.default"; + String DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME = "junit.jupiter.execution.timeout.afterall.method.default"; + Optional getRawConfigurationParameter(String key); boolean isParallelExecutionEnabled(); diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/ExtensionRegistry.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/ExtensionRegistry.java index ea04887586a3..b6eb44633e57 100644 --- a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/ExtensionRegistry.java +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/ExtensionRegistry.java @@ -54,6 +54,7 @@ public class ExtensionRegistry { new DisabledCondition(), // newScriptExecutionCondition(), // new TempDirectory(), // + new TimeoutExtension(), // new RepeatedTestExtension(), // new TestInfoParameterResolver(), // new TestReporterParameterResolver())); diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutConfiguration.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutConfiguration.java new file mode 100644 index 000000000000..b605f5618be3 --- /dev/null +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutConfiguration.java @@ -0,0 +1,109 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.config.JupiterConfiguration.DEFAULT_TIMEOUT_PROPERTY_NAME; + +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.ConcurrentHashMap; +import java.util.function.Supplier; + +import org.junit.jupiter.api.extension.ExtensionContext; +import org.junit.platform.commons.logging.Logger; +import org.junit.platform.commons.logging.LoggerFactory; + +/** + * @since 5.5 + */ +class TimeoutConfiguration { + + private static final Logger logger = LoggerFactory.getLogger(TimeoutConfiguration.class); + + private final TimeoutDurationParser parser = new TimeoutDurationParser(); + private final Map> cache = new ConcurrentHashMap<>(); + private final ExtensionContext extensionContext; + + TimeoutConfiguration(ExtensionContext extensionContext) { + this.extensionContext = extensionContext; + } + + Optional getDefaultTestMethodTimeout() { + return parseOrDefault(DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultTestableMethodTimeout); + } + + Optional getDefaultTestTemplateMethodTimeout() { + return parseOrDefault(DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME, + this::getDefaultTestableMethodTimeout); + } + + Optional getDefaultTestFactoryMethodTimeout() { + return parseOrDefault(DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultTestableMethodTimeout); + } + + Optional getDefaultBeforeAllMethodTimeout() { + return parseOrDefault(DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultLifecycleMethodTimeout); + } + + Optional getDefaultBeforeEachMethodTimeout() { + return parseOrDefault(DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultLifecycleMethodTimeout); + } + + Optional getDefaultAfterEachMethodTimeout() { + return parseOrDefault(DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultLifecycleMethodTimeout); + } + + Optional getDefaultAfterAllMethodTimeout() { + return parseOrDefault(DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultLifecycleMethodTimeout); + } + + private Optional getDefaultTestableMethodTimeout() { + return parseOrDefault(DEFAULT_TESTABLE_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultTimeout); + } + + private Optional getDefaultLifecycleMethodTimeout() { + return parseOrDefault(DEFAULT_LIFECYCLE_METHOD_TIMEOUT_PROPERTY_NAME, this::getDefaultTimeout); + } + + private Optional getDefaultTimeout() { + return parseTimeoutDuration(DEFAULT_TIMEOUT_PROPERTY_NAME); + } + + private Optional parseOrDefault(String propertyName, + Supplier> defaultSupplier) { + Optional timeoutConfiguration = parseTimeoutDuration(propertyName); + return timeoutConfiguration.isPresent() ? timeoutConfiguration : defaultSupplier.get(); + } + + private Optional parseTimeoutDuration(String propertyName) { + return cache.computeIfAbsent(propertyName, key -> extensionContext.getConfigurationParameter(key).map(value -> { + try { + return parser.parse(value); + } + catch (Exception e) { + logger.warn(e, + () -> String.format("Ignored invalid timeout '%s' set via the '%s' configuration parameter.", value, + key)); + return null; + } + })); + } + +} diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDuration.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDuration.java new file mode 100644 index 000000000000..439b2a400451 --- /dev/null +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDuration.java @@ -0,0 +1,71 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import java.util.Objects; +import java.util.concurrent.TimeUnit; + +import org.junit.jupiter.api.Timeout; +import org.junit.platform.commons.util.Preconditions; + +/** + * @since 5.5 + */ +class TimeoutDuration { + + static TimeoutDuration from(Timeout timeout) { + return new TimeoutDuration(timeout.value(), timeout.unit()); + } + + private final long value; + private final TimeUnit unit; + + TimeoutDuration(long value, TimeUnit unit) { + Preconditions.condition(value > 0, () -> "timeout duration must be a positive number: " + value); + this.value = value; + this.unit = Preconditions.notNull(unit, "timeout unit must not be null"); + } + + public long getValue() { + return value; + } + + public TimeUnit getUnit() { + return unit; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + TimeoutDuration that = (TimeoutDuration) o; + return value == that.value && unit == that.unit; + } + + @Override + public int hashCode() { + return Objects.hash(value, unit); + } + + @Override + public String toString() { + String label = unit.name().toLowerCase(); + if (value == 1 && label.endsWith("s")) { + label = label.substring(0, label.length() - 1); + } + return value + " " + label; + } + +} diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDurationParser.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDurationParser.java new file mode 100644 index 000000000000..0a1507b1ff10 --- /dev/null +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutDurationParser.java @@ -0,0 +1,66 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static java.util.concurrent.TimeUnit.DAYS; +import static java.util.concurrent.TimeUnit.HOURS; +import static java.util.concurrent.TimeUnit.MICROSECONDS; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static java.util.concurrent.TimeUnit.MINUTES; +import static java.util.concurrent.TimeUnit.NANOSECONDS; +import static java.util.concurrent.TimeUnit.SECONDS; +import static java.util.regex.Pattern.CASE_INSENSITIVE; +import static java.util.regex.Pattern.UNICODE_CASE; + +import java.time.format.DateTimeParseException; +import java.util.Collections; +import java.util.HashMap; +import java.util.Locale; +import java.util.Map; +import java.util.concurrent.TimeUnit; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * @since 5.5 + */ +class TimeoutDurationParser { + + private static final Pattern PATTERN = Pattern.compile("([1-9]\\d*) ?((?:[nμm]?s)|m|h|d)?", + CASE_INSENSITIVE | UNICODE_CASE); + private static final Map UNITS_BY_ABBREVIATION; + + static { + Map unitsByAbbreviation = new HashMap<>(); + unitsByAbbreviation.put("ns", NANOSECONDS); + unitsByAbbreviation.put("μs", MICROSECONDS); + unitsByAbbreviation.put("ms", MILLISECONDS); + unitsByAbbreviation.put("s", SECONDS); + unitsByAbbreviation.put("m", MINUTES); + unitsByAbbreviation.put("h", HOURS); + unitsByAbbreviation.put("d", DAYS); + UNITS_BY_ABBREVIATION = Collections.unmodifiableMap(unitsByAbbreviation); + } + + TimeoutDuration parse(CharSequence text) throws DateTimeParseException { + Matcher matcher = PATTERN.matcher(text); + if (matcher.matches()) { + long value = Long.parseLong(matcher.group(1)); + String unitAbbreviation = matcher.group(2); + TimeUnit unit = unitAbbreviation == null ? SECONDS + : UNITS_BY_ABBREVIATION.get(unitAbbreviation.toLowerCase(Locale.ENGLISH)); + return new TimeoutDuration(value, unit); + } + throw new DateTimeParseException("Timeout duration is not in the expected format ( [ns|μs|ms|s|m|h|d])", + text, 0); + } + +} diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutExtension.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutExtension.java new file mode 100644 index 000000000000..9c38f5604ef3 --- /dev/null +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutExtension.java @@ -0,0 +1,200 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import java.lang.reflect.AnnotatedElement; +import java.lang.reflect.Method; +import java.util.Optional; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.function.Function; + +import org.junit.jupiter.api.Timeout; +import org.junit.jupiter.api.extension.BeforeAllCallback; +import org.junit.jupiter.api.extension.BeforeEachCallback; +import org.junit.jupiter.api.extension.ExtensionContext; +import org.junit.jupiter.api.extension.ExtensionContext.Store.CloseableResource; +import org.junit.jupiter.api.extension.InvocationInterceptor; +import org.junit.jupiter.api.extension.ReflectiveInvocationContext; +import org.junit.platform.commons.JUnitException; +import org.junit.platform.commons.support.AnnotationSupport; +import org.junit.platform.commons.util.ClassUtils; +import org.junit.platform.commons.util.ReflectionUtils; + +/** + * @since 5.5 + */ +class TimeoutExtension implements BeforeAllCallback, BeforeEachCallback, InvocationInterceptor { + + private static final ExtensionContext.Namespace NAMESPACE = ExtensionContext.Namespace.create(Timeout.class); + private static final String TESTABLE_METHOD_TIMEOUT_KEY = "testable_method_timeout_from_annotation"; + private static final String GLOBAL_TIMEOUT_CONFIG_KEY = "global_timeout_config"; + + @Override + public void beforeAll(ExtensionContext context) { + readAndStoreTimeoutSoChildrenInheritIt(context); + } + + @Override + public void beforeEach(ExtensionContext context) { + readAndStoreTimeoutSoChildrenInheritIt(context); + } + + private void readAndStoreTimeoutSoChildrenInheritIt(ExtensionContext context) { + readTimeoutFromAnnotation(context.getElement()).ifPresent( + timeout -> context.getStore(NAMESPACE).put(TESTABLE_METHOD_TIMEOUT_KEY, timeout)); + } + + @Override + public void interceptBeforeAllMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + interceptLifecycleMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultBeforeAllMethodTimeout); + } + + @Override + public void interceptBeforeEachMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + interceptLifecycleMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultBeforeEachMethodTimeout); + } + + @Override + public void interceptTestMethod(Invocation invocation, ReflectiveInvocationContext invocationContext, + ExtensionContext extensionContext) throws Throwable { + interceptTestableMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultTestMethodTimeout); + } + + @Override + public void interceptTestTemplateMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + interceptTestableMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultTestTemplateMethodTimeout); + } + + @Override + public T interceptTestFactoryMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + return interceptTestableMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultTestFactoryMethodTimeout); + } + + @Override + public void interceptAfterEachMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + interceptLifecycleMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultAfterEachMethodTimeout); + } + + @Override + public void interceptAfterAllMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) throws Throwable { + interceptLifecycleMethod(invocation, invocationContext, extensionContext, + TimeoutConfiguration::getDefaultAfterAllMethodTimeout); + } + + private void interceptLifecycleMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext, + TimeoutProvider defaultTimeoutProvider) throws Throwable { + TimeoutDuration timeout = readTimeoutFromAnnotation(Optional.of(invocationContext.getExecutable())).orElse( + null); + intercept(invocation, invocationContext, extensionContext, timeout, defaultTimeoutProvider); + } + + @SuppressWarnings("OptionalUsedAsFieldOrParameterType") + private Optional readTimeoutFromAnnotation(Optional element) { + return AnnotationSupport.findAnnotation(element, Timeout.class).map(TimeoutDuration::from); + } + + private T interceptTestableMethod(Invocation invocation, + ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext, + TimeoutProvider defaultTimeoutProvider) throws Throwable { + TimeoutDuration timeout = extensionContext.getStore(NAMESPACE).get(TESTABLE_METHOD_TIMEOUT_KEY, + TimeoutDuration.class); + return intercept(invocation, invocationContext, extensionContext, timeout, defaultTimeoutProvider); + } + + private T intercept(Invocation invocation, ReflectiveInvocationContext invocationContext, + ExtensionContext extensionContext, TimeoutDuration explicitTimeout, TimeoutProvider defaultTimeoutProvider) + throws Throwable { + TimeoutDuration timeout = explicitTimeout == null ? getDefaultTimeout(extensionContext, defaultTimeoutProvider) + : explicitTimeout; + return decorate(invocation, invocationContext, extensionContext, timeout).proceed(); + } + + private TimeoutDuration getDefaultTimeout(ExtensionContext extensionContext, + TimeoutProvider defaultTimeoutProvider) { + return defaultTimeoutProvider.apply(getGlobalTimeoutConfiguration(extensionContext)).orElse(null); + } + + private TimeoutConfiguration getGlobalTimeoutConfiguration(ExtensionContext extensionContext) { + ExtensionContext root = extensionContext.getRoot(); + return root.getStore(NAMESPACE).getOrComputeIfAbsent(GLOBAL_TIMEOUT_CONFIG_KEY, + key -> new TimeoutConfiguration(root), TimeoutConfiguration.class); + } + + private Invocation decorate(Invocation invocation, ReflectiveInvocationContext invocationContext, + ExtensionContext extensionContext, TimeoutDuration timeout) { + if (timeout == null) { + return invocation; + } + return new TimeoutInvocation<>(invocation, timeout, getExecutor(extensionContext), + () -> describe(invocationContext, extensionContext)); + } + + private String describe(ReflectiveInvocationContext invocationContext, ExtensionContext extensionContext) { + Method method = invocationContext.getExecutable(); + Optional> testClass = extensionContext.getTestClass(); + if (testClass.isPresent() && invocationContext.getTargetClass().equals(testClass.get())) { + return String.format("%s(%s)", method.getName(), ClassUtils.nullSafeToString(method.getParameterTypes())); + } + return ReflectionUtils.getFullyQualifiedMethodName(invocationContext.getTargetClass(), method); + } + + private ScheduledExecutorService getExecutor(ExtensionContext extensionContext) { + return extensionContext.getRoot().getStore(NAMESPACE).getOrComputeIfAbsent(ExecutorResource.class).get(); + } + + @FunctionalInterface + private interface TimeoutProvider extends Function> { + } + + private static class ExecutorResource implements CloseableResource { + + private final ScheduledExecutorService executor; + + ExecutorResource() { + executor = Executors.newSingleThreadScheduledExecutor(runnable -> { + Thread thread = new Thread(runnable, "junit-jupiter-timeout-watcher"); + thread.setPriority(Thread.MAX_PRIORITY); + return thread; + }); + } + + ScheduledExecutorService get() { + return executor; + } + + @Override + public void close() throws Throwable { + executor.shutdown(); + boolean terminated = executor.awaitTermination(5, TimeUnit.SECONDS); + if (!terminated) { + executor.shutdownNow(); + throw new JUnitException("Scheduled executor could not be stopped in an orderly manner"); + } + } + + } + +} diff --git a/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutInvocation.java b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutInvocation.java new file mode 100644 index 000000000000..8d81b2e5d219 --- /dev/null +++ b/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/extension/TimeoutInvocation.java @@ -0,0 +1,94 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeoutException; +import java.util.function.Supplier; + +import org.junit.jupiter.api.extension.InvocationInterceptor.Invocation; +import org.junit.platform.commons.util.BlacklistedExceptions; + +/** + * @since 5.5 + */ +class TimeoutInvocation implements Invocation { + + private final Invocation delegate; + private final TimeoutDuration timeout; + private final ScheduledExecutorService executor; + private final Supplier descriptionSupplier; + + TimeoutInvocation(Invocation delegate, TimeoutDuration timeout, ScheduledExecutorService executor, + Supplier descriptionSupplier) { + this.delegate = delegate; + this.timeout = timeout; + this.executor = executor; + this.descriptionSupplier = descriptionSupplier; + } + + @Override + public T proceed() throws Throwable { + InterruptTask interruptTask = new InterruptTask(Thread.currentThread()); + ScheduledFuture future = executor.schedule(interruptTask, timeout.getValue(), timeout.getUnit()); + Throwable failure = null; + T result = null; + try { + result = delegate.proceed(); + } + catch (Throwable t) { + BlacklistedExceptions.rethrowIfBlacklisted(t); + failure = t; + } + finally { + boolean cancelled = future.cancel(false); + if (!cancelled) { + future.get(); + } + if (interruptTask.executed) { + Thread.interrupted(); + failure = createTimeoutException(failure); + } + } + if (failure != null) { + throw failure; + } + return result; + } + + private TimeoutException createTimeoutException(Throwable failure) { + String message = String.format("%s timed out after %s", descriptionSupplier.get(), timeout); + TimeoutException timeoutError = new TimeoutException(message); + if (failure != null) { + timeoutError.addSuppressed(failure); + } + return timeoutError; + } + + static class InterruptTask implements Runnable { + + private final Thread thread; + private volatile boolean executed; + + InterruptTask(Thread thread) { + this.thread = thread; + } + + @Override + public void run() { + executed = true; + thread.interrupt(); + } + + } + +} diff --git a/junit-jupiter-engine/src/module/org.junit.jupiter.engine/module-info.java b/junit-jupiter-engine/src/module/org.junit.jupiter.engine/module-info.java index 85e0c0715b37..8a71502fbdb9 100644 --- a/junit-jupiter-engine/src/module/org.junit.jupiter.engine/module-info.java +++ b/junit-jupiter-engine/src/module/org.junit.jupiter.engine/module-info.java @@ -20,4 +20,6 @@ provides org.junit.platform.engine.TestEngine with org.junit.jupiter.engine.JupiterTestEngine; + + opens org.junit.jupiter.engine.extension to org.junit.platform.commons; } diff --git a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/ExtensionRegistryTests.java b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/ExtensionRegistryTests.java index 440a02a18d25..a84bba33d1ec 100644 --- a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/ExtensionRegistryTests.java +++ b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/ExtensionRegistryTests.java @@ -29,6 +29,7 @@ import org.junit.jupiter.api.extension.BeforeEachCallback; import org.junit.jupiter.api.extension.ExecutionCondition; import org.junit.jupiter.api.extension.Extension; +import org.junit.jupiter.api.extension.InvocationInterceptor; import org.junit.jupiter.api.extension.ParameterResolver; import org.junit.jupiter.api.extension.TestTemplateInvocationContextProvider; import org.junit.jupiter.engine.config.JupiterConfiguration; @@ -40,7 +41,7 @@ */ class ExtensionRegistryTests { - private static final int NUM_DEFAULT_EXTENSIONS = 6; + private static final int NUM_DEFAULT_EXTENSIONS = 7; private final JupiterConfiguration configuration = mock(JupiterConfiguration.class); @@ -63,10 +64,10 @@ void newRegistryWithoutParentHasDefaultExtensionsPlusAutodetectedExtensionsLoade List extensions = registry.getExtensions(Extension.class); assertEquals(NUM_DEFAULT_EXTENSIONS + 1, extensions.size()); - assertDefaultGlobalExtensionsAreRegistered(2); + assertDefaultGlobalExtensionsAreRegistered(3); assertExtensionRegistered(registry, ServiceLoaderExtension.class); - assertEquals(2, countExtensions(registry, BeforeAllCallback.class)); + assertEquals(3, countExtensions(registry, BeforeAllCallback.class)); } @Test @@ -155,22 +156,24 @@ private void assertExtensionRegistered(ExtensionRegistry registry, Class parsesNumbersWithUnits() { + var unitsWithRepresentations = Map.of( // + NANOSECONDS, "ns", // + MICROSECONDS, "μs", // + MILLISECONDS, "ms", // + SECONDS, "s", // + MINUTES, "m", // + HOURS, "h", // + DAYS, "d"); + return unitsWithRepresentations.entrySet().stream() // + .map(entry -> { + var unit = entry.getKey(); + var plainRepresentation = entry.getValue(); + var representations = Stream.of( // + plainRepresentation, // + " " + plainRepresentation, // + plainRepresentation.toUpperCase(), // + " " + plainRepresentation.toUpperCase()); + return dynamicContainer(unit.name().toLowerCase(), + representations.map(representation -> dynamicTest("\"" + representation + "\"", () -> { + var expected = new TimeoutDuration(42, unit); + var actual = parser.parse("42" + representation); + assertEquals(expected, actual); + }))); + }); + } + + @Test + void rejectsNumbersStartingWithZero() { + assertThrows(DateTimeParseException.class, () -> parser.parse("01")); + } +} diff --git a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutDurationTests.java b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutDurationTests.java new file mode 100644 index 000000000000..ccae04ddba94 --- /dev/null +++ b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutDurationTests.java @@ -0,0 +1,43 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static java.util.concurrent.TimeUnit.MINUTES; +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.assertj.core.api.Assertions.assertThat; + +import org.junit.jupiter.api.Test; + +/** + * @since 5.5 + */ +class TimeoutDurationTests { + + @Test + void formatsDurationNicely() { + assertThat(new TimeoutDuration(1, SECONDS)).hasToString("1 second"); + assertThat(new TimeoutDuration(2, SECONDS)).hasToString("2 seconds"); + } + + @Test + void fulfillsEqualsAndHashCodeContract() { + var oneSecond = new TimeoutDuration(1, SECONDS); + + assertThat(oneSecond) // + .isEqualTo(oneSecond) // + .isEqualTo(new TimeoutDuration(1, SECONDS)) // + .hasSameHashCodeAs(new TimeoutDuration(1, SECONDS)) // + .isNotEqualTo("foo") // + .isNotEqualTo(new TimeoutDuration(2, SECONDS)) // + .isNotEqualTo(new TimeoutDuration(1, MINUTES)); + } + +} diff --git a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutExtensionTests.java b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutExtensionTests.java new file mode 100644 index 000000000000..c1a96fb3fbfd --- /dev/null +++ b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutExtensionTests.java @@ -0,0 +1,492 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static java.util.concurrent.TimeUnit.NANOSECONDS; +import static java.util.stream.Collectors.toList; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.DynamicTest.dynamicTest; +import static org.junit.jupiter.engine.Constants.DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.jupiter.engine.Constants.DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME; +import static org.junit.platform.commons.util.CollectionUtils.getOnlyElement; +import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass; +import static org.junit.platform.engine.discovery.DiscoverySelectors.selectMethod; +import static org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder.request; + +import java.time.Duration; +import java.util.Map; +import java.util.concurrent.TimeoutException; +import java.util.stream.Stream; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.DynamicTest; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.RepeatedTest; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestFactory; +import org.junit.jupiter.api.Timeout; +import org.junit.jupiter.engine.AbstractJupiterTestEngineTests; +import org.junit.platform.commons.PreconditionViolationException; +import org.junit.platform.testkit.engine.EngineExecutionResults; +import org.junit.platform.testkit.engine.Events; +import org.junit.platform.testkit.engine.Execution; + +/** + * @since 5.5 + */ +@DisplayName("@Timeout") +class TimeoutExtensionTests extends AbstractJupiterTestEngineTests { + + @Test + @DisplayName("is applied on annotated @Test methods") + void appliesTimeoutOnAnnotatedTestMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectMethod(TimeoutAnnotatedTestMethodTestCase.class, "testMethod")) // + .configurationParameter(DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.tests(), "testMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("testMethod() timed out after 10 milliseconds"); + } + + @Test + @DisplayName("is applied on annotated @TestTemplate methods") + void appliesTimeoutOnAnnotatedTestTemplateMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectMethod(TimeoutAnnotatedTestMethodTestCase.class, "testTemplateMethod")) // + .configurationParameter(DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Stream.of("repetition 1", "repetition 2").forEach(displayName -> { + Execution execution = findExecution(results.tests(), displayName); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("testTemplateMethod() timed out after 10 milliseconds"); + }); + } + + @Test + @DisplayName("is applied on annotated @TestFactory methods") + void appliesTimeoutOnAnnotatedTestFactoryMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectMethod(TimeoutAnnotatedTestMethodTestCase.class, "testFactoryMethod")) // + .configurationParameter(DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.containers(), "testFactoryMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("testFactoryMethod() timed out after 10 milliseconds"); + } + + @TestFactory + @DisplayName("is applied on testable methods in annotated classes") + Stream appliesTimeoutOnTestableMethodsInAnnotatedClasses() { + return Stream.of(TimeoutAnnotatedClassTestCase.class, InheritedTimeoutAnnotatedClassTestCase.class).map( + testClass -> dynamicTest(testClass.getSimpleName(), () -> { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(testClass)) // + .configurationParameter(DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .configurationParameter(DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .configurationParameter(DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Stream.of("testMethod()", "repetition 1", "repetition 2", "testFactoryMethod()").forEach( + displayName -> { + Execution execution = findExecution(results.all(), displayName); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessageEndingWith("timed out after 10000000 nanoseconds"); + }); + })); + } + + @Test + @DisplayName("fails uninterruptible methods") + void failsUninterruptibleMethods() { + EngineExecutionResults results = executeTestsForClass(UninterruptibleMethodTestCase.class); + + Execution execution = findExecution(results.tests(), "uninterruptibleMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("uninterruptibleMethod() timed out after 1 millisecond"); + } + + @Test + @DisplayName("is applied on annotated @BeforeAll methods") + void appliesTimeoutOnAnnotatedBeforeAllMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(TimeoutAnnotatedBeforeAllMethodTestCase.class)) // + .configurationParameter(DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.containers(), + TimeoutAnnotatedBeforeAllMethodTestCase.class.getSimpleName()); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("setUp() timed out after 10 milliseconds"); + } + + @Test + @DisplayName("is applied on annotated @BeforeEach methods") + void appliesTimeoutOnAnnotatedBeforeEachMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(TimeoutAnnotatedBeforeEachMethodTestCase.class)) // + .configurationParameter(DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.tests(), "testMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("setUp() timed out after 10 milliseconds"); + } + + @Test + @DisplayName("is applied on annotated @AfterEach methods") + void appliesTimeoutOnAnnotatedAfterEachMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(TimeoutAnnotatedAfterEachMethodTestCase.class)) // + .configurationParameter(DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.tests(), "testMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("tearDown() timed out after 10 milliseconds"); + } + + @Test + @DisplayName("is applied on annotated @AfterAll methods") + void appliesTimeoutOnAnnotatedAfterAllMethods() { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(TimeoutAnnotatedAfterAllMethodTestCase.class)) // + .configurationParameter(DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME, "42ns") // + .build()); + + Execution execution = findExecution(results.containers(), + TimeoutAnnotatedAfterAllMethodTestCase.class.getSimpleName()); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessage("tearDown() timed out after 10 milliseconds"); + } + + @TestFactory + @DisplayName("is applied from configuration parameters by default") + Stream appliesDefaultTimeoutsFromConfigurationParameters() { + return Map.of(DEFAULT_BEFORE_ALL_METHOD_TIMEOUT_PROPERTY_NAME, "beforeAll()", // + DEFAULT_BEFORE_EACH_METHOD_TIMEOUT_PROPERTY_NAME, "beforeEach()", // + DEFAULT_TEST_METHOD_TIMEOUT_PROPERTY_NAME, "test()", // + DEFAULT_TEST_TEMPLATE_METHOD_TIMEOUT_PROPERTY_NAME, "testTemplate()", // + DEFAULT_TEST_FACTORY_METHOD_TIMEOUT_PROPERTY_NAME, "testFactory()", // + DEFAULT_AFTER_EACH_METHOD_TIMEOUT_PROPERTY_NAME, "afterEach()", // + DEFAULT_AFTER_ALL_METHOD_TIMEOUT_PROPERTY_NAME, "afterAll()" // + ).entrySet().stream().map(entry -> dynamicTest("uses " + entry.getKey() + " config param", () -> { + EngineExecutionResults results = executeTests(request() // + .selectors(selectClass(PlainTestCase.class)) // + .configurationParameter(entry.getKey(), "1ns") // + .build()); + var failure = results.all().executions().failed() // + .map(execution -> execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .findFirst(); + assertThat(failure).containsInstanceOf(TimeoutException.class); + assertThat(failure.get()).hasMessage(entry.getValue() + " timed out after 1 nanosecond"); + })); + } + + @Test + @DisplayName("does not swallow blacklisted exceptions") + void doesNotSwallowBlacklistedExceptions() { + assertThrows(OutOfMemoryError.class, () -> executeTestsForClass(BlacklistedExceptionTestCase.class)); + } + + @Test + @DisplayName("does not affect tests that don't exceed the timeout") + void doesNotAffectTestsThatDoNotExceedTimeoutDuration() { + var results = executeTestsForClass(NonTimeoutExceedingTestCase.class); + results.all().assertStatistics(stats -> stats.failed(0)); + } + + @Test + @DisplayName("includes fully qualified class name if method is not in the test class") + void includesClassNameIfMethodIsNotInTestClass() { + EngineExecutionResults results = executeTestsForClass(NestedClassWithOuterSetupMethodTestCase.class); + + Execution execution = findExecution(results.tests(), "testMethod()"); + assertThat(execution.getDuration()) // + .isGreaterThanOrEqualTo(Duration.ofMillis(10)) // + .isLessThan(Duration.ofSeconds(1)); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(TimeoutException.class) // + .hasMessageEndingWith( + "$NestedClassWithOuterSetupMethodTestCase#setUp() timed out after 10 milliseconds"); + } + + @Test + @DisplayName("reports illegal timeout durations") + void reportsIllegalTimeoutDurations() { + EngineExecutionResults results = executeTestsForClass(IllegalTimeoutDurationTestCase.class); + + Execution execution = findExecution(results.tests(), "testMethod()"); + assertThat(execution.getTerminationInfo().getExecutionResult().getThrowable().orElseThrow()) // + .isInstanceOf(PreconditionViolationException.class) // + .hasMessage("timeout duration must be a positive number: 0"); + } + + private Execution findExecution(Events events, String displayName) { + return getOnlyElement(events // + .executions() // + .filter(execution -> execution.getTestDescriptor().getDisplayName().contains(displayName)) // + .collect(toList())); + } + + static class TimeoutAnnotatedTestMethodTestCase { + @Test + @Timeout(value = 10, unit = MILLISECONDS) + void testMethod() throws Exception { + Thread.sleep(1000); + } + + @RepeatedTest(2) + @Timeout(value = 10, unit = MILLISECONDS) + void testTemplateMethod() throws Exception { + Thread.sleep(1000); + } + + @TestFactory + @Timeout(value = 10, unit = MILLISECONDS) + Stream testFactoryMethod() throws Exception { + Thread.sleep(1000); + return Stream.empty(); + } + } + + static class TimeoutAnnotatedBeforeAllMethodTestCase { + @BeforeAll + @Timeout(value = 10, unit = MILLISECONDS) + static void setUp() throws Exception { + Thread.sleep(1000); + } + + @Test + void testMethod() { + // never called + } + } + + static class TimeoutAnnotatedBeforeEachMethodTestCase { + @BeforeEach + @Timeout(value = 10, unit = MILLISECONDS) + void setUp() throws Exception { + Thread.sleep(1000); + } + + @Test + void testMethod() { + // never called + } + } + + static class TimeoutAnnotatedAfterEachMethodTestCase { + @Test + void testMethod() { + // do nothing + } + + @AfterEach + @Timeout(value = 10, unit = MILLISECONDS) + void tearDown() throws Exception { + Thread.sleep(1000); + } + } + + static class TimeoutAnnotatedAfterAllMethodTestCase { + @Test + void testMethod() { + // do nothing + } + + @AfterAll + @Timeout(value = 10, unit = MILLISECONDS) + static void tearDown() throws Exception { + Thread.sleep(1000); + } + } + + @Timeout(value = 10_000_000, unit = NANOSECONDS) + static class TimeoutAnnotatedClassTestCase { + @Nested + class NestedClass { + @Test + void testMethod() throws Exception { + Thread.sleep(1000); + } + + @RepeatedTest(2) + void testTemplateMethod() throws Exception { + Thread.sleep(1000); + } + + @TestFactory + Stream testFactoryMethod() throws Exception { + Thread.sleep(1000); + return Stream.empty(); + } + } + } + + static class InheritedTimeoutAnnotatedClassTestCase extends TimeoutAnnotatedClassTestCase { + } + + static class UninterruptibleMethodTestCase { + @Test + @Timeout(value = 1, unit = MILLISECONDS) + void uninterruptibleMethod() { + new UninterruptibleInvocation(50, MILLISECONDS).proceed(); + } + } + + static class PlainTestCase { + @BeforeAll + static void beforeAll() throws Exception { + Thread.sleep(10); + } + + @BeforeEach + void beforeEach() throws Exception { + Thread.sleep(10); + } + + @Test + void test() throws Exception { + Thread.sleep(10); + } + + @RepeatedTest(2) + void testTemplate() throws Exception { + Thread.sleep(10); + } + + @TestFactory + Stream testFactory() throws Exception { + Thread.sleep(10); + return Stream.empty(); + } + + @AfterEach + void afterEach() throws Exception { + Thread.sleep(10); + } + + @AfterAll + static void afterAll() throws Exception { + Thread.sleep(10); + } + } + + static class BlacklistedExceptionTestCase { + @Test + @Timeout(value = 1, unit = NANOSECONDS) + void test() { + new UninterruptibleInvocation(10, MILLISECONDS).proceed(); + throw new OutOfMemoryError(); + } + } + + @Timeout(10) + static class NonTimeoutExceedingTestCase { + @Test + void testMethod() { + } + + @RepeatedTest(1) + void testTemplateMethod() { + } + + @TestFactory + Stream testFactoryMethod() { + return Stream.of(dynamicTest("dynamicTest", () -> { + })); + } + } + + static class NestedClassWithOuterSetupMethodTestCase { + + @Timeout(value = 10, unit = MILLISECONDS) + @BeforeEach + void setUp() throws Exception { + Thread.sleep(1000); + } + + @Nested + class NestedClass { + + @BeforeEach + void setUp() { + } + + @Test + void testMethod() { + } + + } + + } + + static class IllegalTimeoutDurationTestCase { + + @Test + @Timeout(0) + void testMethod() { + } + + } + +} diff --git a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutInvocationTests.java b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutInvocationTests.java new file mode 100644 index 000000000000..87489d739aca --- /dev/null +++ b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/TimeoutInvocationTests.java @@ -0,0 +1,56 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static java.util.concurrent.TimeUnit.NANOSECONDS; +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeoutException; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.function.ThrowingConsumer; + +/** + * @since 5.5 + */ +class TimeoutInvocationTests { + + @Test + void resetsInterruptFlag() { + var exception = assertThrows(TimeoutException.class, () -> withExecutor(executor -> { + var uninterruptibleInvocation = new UninterruptibleInvocation(100, MILLISECONDS); + var duration = new TimeoutDuration(1, NANOSECONDS); + var timeoutInvocation = new TimeoutInvocation<>(uninterruptibleInvocation, duration, executor, + () -> "execution"); + timeoutInvocation.proceed(); + })); + assertFalse(Thread.currentThread().isInterrupted()); + assertThat(exception).hasMessage("execution timed out after 1 nanosecond"); + } + + private void withExecutor(ThrowingConsumer consumer) throws Throwable { + ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor(); + try { + consumer.accept(executor); + } + finally { + executor.shutdown(); + assertTrue(executor.awaitTermination(5, SECONDS)); + } + } +} diff --git a/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/UninterruptibleInvocation.java b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/UninterruptibleInvocation.java new file mode 100644 index 000000000000..d4e47dd0df02 --- /dev/null +++ b/junit-jupiter-engine/src/test/java/org/junit/jupiter/engine/extension/UninterruptibleInvocation.java @@ -0,0 +1,46 @@ +/* + * Copyright 2015-2019 the original author or authors. + * + * All rights reserved. This program and the accompanying materials are + * made available under the terms of the Eclipse Public License v2.0 which + * accompanies this distribution and is available at + * + * https://www.eclipse.org/legal/epl-v20.html + */ + +package org.junit.jupiter.engine.extension; + +import static java.util.concurrent.TimeUnit.NANOSECONDS; +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.concurrent.TimeUnit; +import java.util.stream.IntStream; + +import org.junit.jupiter.api.extension.InvocationInterceptor.Invocation; + +/** + * @since 5.5 + */ +class UninterruptibleInvocation implements Invocation { + + private final long duration; + private final TimeUnit unit; + + UninterruptibleInvocation(long duration, TimeUnit unit) { + this.duration = duration; + this.unit = unit; + } + + @Override + public Void proceed() { + long startTime = System.nanoTime(); + while (true) { + assertThat(IntStream.range(1, 1_000_000).sum()).isGreaterThan(0); + long elapsedTime = System.nanoTime() - startTime; + if (elapsedTime > NANOSECONDS.convert(duration, unit)) { + return null; + } + } + } + +} diff --git a/platform-tooling-support-tests/platform-tooling-support-tests.gradle.kts b/platform-tooling-support-tests/platform-tooling-support-tests.gradle.kts index 9297cc7a9dc4..cc03fd5e3bfc 100644 --- a/platform-tooling-support-tests/platform-tooling-support-tests.gradle.kts +++ b/platform-tooling-support-tests/platform-tooling-support-tests.gradle.kts @@ -22,6 +22,9 @@ dependencies { testImplementation("com.tngtech.archunit:archunit-junit5-api:${Versions.archunit}") { because("checking the architecture of JUnit 5") } + testImplementation("org.codehaus.groovy:groovy-all:${Versions.groovy}") { + because("it provides convenience methods to handle process output") + } testRuntimeOnly("com.tngtech.archunit:archunit-junit5-engine:${Versions.archunit}") { because("contains the ArchUnit TestEngine implementation") } diff --git a/platform-tooling-support-tests/projects/jar-describe-module/junit-jupiter-engine.expected.txt b/platform-tooling-support-tests/projects/jar-describe-module/junit-jupiter-engine.expected.txt index 04a24c37be2f..c0845a305c27 100644 --- a/platform-tooling-support-tests/projects/jar-describe-module/junit-jupiter-engine.expected.txt +++ b/platform-tooling-support-tests/projects/jar-describe-module/junit-jupiter-engine.expected.txt @@ -6,4 +6,5 @@ requires org.junit.jupiter.api transitive requires org.junit.platform.engine transitive requires org.opentest4j transitive provides org.junit.platform.engine.TestEngine with org.junit.jupiter.engine.JupiterTestEngine +qualified opens org.junit.jupiter.engine.extension to org.junit.platform.commons contains org.junit.jupiter.engine diff --git a/platform-tooling-support-tests/src/test/java/platform/tooling/support/tests/ModularUserGuideTests.java b/platform-tooling-support-tests/src/test/java/platform/tooling/support/tests/ModularUserGuideTests.java index 86cc0e9e56a4..9ee4baaffad3 100644 --- a/platform-tooling-support-tests/src/test/java/platform/tooling/support/tests/ModularUserGuideTests.java +++ b/platform-tooling-support-tests/src/test/java/platform/tooling/support/tests/ModularUserGuideTests.java @@ -13,6 +13,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertLinesMatch; import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; import java.io.File; import java.io.PrintWriter; @@ -24,6 +25,7 @@ import java.util.List; import java.util.spi.ToolProvider; +import org.codehaus.groovy.runtime.ProcessGroovyMethods; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; import platform.tooling.support.Helper; @@ -138,11 +140,15 @@ private static List junit(Path temp, Writer out, Writer err) throws Exce // command.forEach(System.out::println); var builder = new ProcessBuilder(command).directory(temp.toFile()); - // var java = builder.inheritIO().start(); // show "console" output and errors - var java = builder.redirectErrorStream(true).redirectOutput(ProcessBuilder.Redirect.DISCARD).start(); - var code = java.waitFor(); - - assertEquals(0, code, out.toString()); + var java = builder.start(); + ProcessGroovyMethods.waitForProcessOutput(java, out, err); + var code = java.exitValue(); + + if (code != 0) { + System.out.println(out); + System.err.println(err); + fail("Unexpected exit code: " + code); + } return command; }