Document declarative timeouts in User Guide

Author: marcphilipp

documentation/src/docs/asciidoc/release-notes/release-notes-5.5.0-M2.adoc

@@ -51,6 +51,8 @@ on GitHub.
 
 ==== New Features and Improvements
 
+* Support for declarative timeouts using `@Timeout` or configuration parameters (see
+  <<../user-guide/index.adoc#writing-tests-declarative-timeouts, User Guide>> for details)
 * New overloaded variants of `Assertions.assertLinesMatch(...)` that accept a `String` or
   a `Supplier<String>` for a custom failure message.
 * Failure messages for `Assertions.assertLinesMatch(...)` now emit each expected and

documentation/src/docs/asciidoc/user-guide/writing-tests.adoc

@@ -40,6 +40,7 @@ in the `junit-jupiter-api` module.
 | `@Nested`                | Denotes that the annotated class is a non-static <<writing-tests-nested,nested test class>>. `@BeforeAll` and `@AfterAll` methods cannot be used directly in a `@Nested` test class unless the "per-class" <<writing-tests-test-instance-lifecycle, test instance lifecycle>> is used. Such annotations are not _inherited_.
 | `@Tag`                   | Used to declare <<writing-tests-tagging-and-filtering,tags for filtering tests>>, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4. Such annotations are _inherited_ at the class level but not at the method level.
 | `@Disabled`              | Used to <<writing-tests-disabling,disable>> a test class or test method; analogous to JUnit 4's `@Ignore`. Such annotations are not _inherited_.
+| `@Timeout`               | Used to fail a test, test factory, test template, or lifecycle method if its execution exceeds a given duration. Such annotations are _inherited_.
 | `@ExtendWith`            | Used to <<extensions-registration-declarative,register extensions declaratively>>. Such annotations are _inherited_.
 | `@RegisterExtension`     | Used to <<extensions-registration-programmatic,register extensions programmatically>> via fields. Such fields are _inherited_ unless they are _shadowed_.
 | `@TempDir`               | Used to supply a <<writing-tests-built-in-extensions-TempDirectory,temporary directory>> via field injection or parameter injection in a lifecycle method or test method; located in the `org.junit.jupiter.api.io` package.
@@ -217,11 +218,12 @@ include::{testDir}/example/AssertionsDemo.java[tags=user_guide]
 
 [[writing-tests-assertions-preemptive-timeouts]]
 [WARNING]
-.Preemptive Timeouts
+.Preemptive Timeouts with `assertTimeoutPreemptively()`
 ====
-The various `assertTimeoutPreemptively()` methods in the `Assertions` class execute the
-provided `executable` or `supplier` in a different thread than that of the calling code.
-This behavior can lead to undesirable side effects if the code that is executed within the
+Contrary to <<writing-tests-declarative-timeouts, declarative timeouts>>, the various
+`assertTimeoutPreemptively()` methods in the `Assertions` class execute the provided
+`executable` or `supplier` in a different thread than that of the calling code. This
+behavior can lead to undesirable side effects if the code that is executed within the
 `executable` or `supplier` relies on `java.lang.ThreadLocal` storage.
 
 One common example of this is the transactional testing support in the Spring Framework.
@@ -1570,6 +1572,95 @@ include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide]
 ----
 
 
+[[writing-tests-declarative-timeouts]]
+=== Timeouts
+
+.Declarative timeouts are an experimental feature
+WARNING: You're invited to give it a try and provide feedback to the JUnit team so they
+can improve and eventually <<api-evolution, promote>> this feature.
+
+The `@Timeout` annotation allows to declare that a test, test factory, test template, or
+lifecycle method should fail if its execution time exceeds a given duration, and
+optionally a time unit (seconds are used by default).
+
+The following example shows how `@Timeout` is applied to lifecycle and test methods.
+
+[source,java]
+----
+include::{testDir}/example/TimeoutDemo.java[tags=user_guide]
+----
+
+Contrary to the `assertTimeoutPreemptively()` assertion, the execution of the annotated
+method proceeds in the main thread of the test. If the timeout is exceeded, the main
+thread is interrupted from another thread. This is done to ensure interoperability with
+frameworks such as Spring that make extensive use of mechanisms that are sensitive to the
+currently running thread (e.g. `ThreadLocals`).
+
+To apply the same timeout to all test methods within a test class and all its `@Nested`
+classes, you can declare the `@Timeout` annotation on the class level. It will then be
+applied to all test, test factory, and test template methods within that class and its
+`@Nested` classes unless overridden by a `@Timeout` annotation on a method or `@Nested`
+class. Please note that `@Timeout` annotations declared on the class level are not
+applied to lifecycle methods.
+
+Declaring `@Timeout` on a test factory method checks that the method returns within the
+specified duration but does verify the execution time of the returned `DynamicTests`.
+Please use `assertTimeout()`/`assertTimeoutPreemptively()` for that purpose.
+
+If `@Timeout` is present on a test template method, such as a method annotated with
+`@RepeatedTest` or `@ParameterizedTest`, each invocation is checked to finish within the
+given timeout.
+
+The following <<running-tests-config-params, configuration parameters>> can be used to
+specify global timeouts for all methods of a certain category unless they or an enclosing
+test class is annotated with `@Timeout`:
+
+`junit.jupiter.execution.timeout.default`::
+    Default timeout for all testable and lifecycle methods
+`junit.jupiter.execution.timeout.testable.method.default`::
+    Default timeout for all testable methods
+`junit.jupiter.execution.timeout.test.method.default`::
+    Default timeout for `@Test` methods
+`junit.jupiter.execution.timeout.testtemplate.method.default`::
+    Default timeout for `@TestTemplate` methods
+`junit.jupiter.execution.timeout.testfactory.method.default`::
+    Default timeout for `@TestFactory` methods
+`junit.jupiter.execution.timeout.lifecycle.method.default`::
+    Default timeout for all lifecycle methods
+`junit.jupiter.execution.timeout.beforeall.method.default`::
+    Default timeout for `@BeforeAll` methods
+`junit.jupiter.execution.timeout.beforeeach.method.default`::
+    Default timeout for `@BeforeEach` methods
+`junit.jupiter.execution.timeout.aftereach.method.default`::
+    Default timeout for `@AfterEach` methods
+`junit.jupiter.execution.timeout.afterall.method.default`::
+    Default timeout for `@AfterAll` methods
+
+More specific configuration parameters override less specific ones. For example,
+`junit.jupiter.execution.timeout.test.method.default` overrides
+`junit.jupiter.execution.timeout.testable.method.default` which overrides
+`junit.jupiter.execution.timeout.default`.
+
+The values of such configuration parameters must be in the following, case-insensitive
+format: `<number> [ns|μs|ms|s|m|h|d]`. The space between the number and the unit may be
+omitted. Specifying no unit is equivalent to using seconds.
+
+.Example timeout configuration parameter values
+[cols="20,80"]
+|===
+| Parameter value   | Equivalent annotation
+
+| `42`              | `@Timeout(42)`
+| `42 ns`           | `@Timeout(value = 42, unit = NANOSECONDS)`
+| `42 μs`           | `@Timeout(value = 42, unit = MICROSECONDS)`
+| `42 ms`           | `@Timeout(value = 42, unit = MILLISECONDS)`
+| `42 s`            | `@Timeout(value = 42, unit = SECONDS)`
+| `42 m`            | `@Timeout(value = 42, unit = MINUTES)`
+| `42 h`            | `@Timeout(value = 42, unit = HOURS)`
+| `42 d`            | `@Timeout(value = 42, unit = DAYS)`
+|===
+
+
 [[writing-tests-parallel-execution]]
 === Parallel Execution
 

documentation/src/test/java/example/TimeoutDemo.java

@@ -0,0 +1,35 @@
+/*
+ * Copyright 2015-2019 the original author or authors.
+ *
+ * All rights reserved. This program and the accompanying materials are
+ * made available under the terms of the Eclipse Public License v2.0 which
+ * accompanies this distribution and is available at
+ *
+ * https://www.eclipse.org/legal/epl-v20.html
+ */
+
+package example;
+
+import java.util.concurrent.TimeUnit;
+
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Timeout;
+
+// tag::user_guide[]
+class TimeoutDemo {
+
+	@BeforeEach
+	@Timeout(5)
+	void setUp() {
+		// fails if execution time exceeds 5 seconds
+	}
+
+	@Test
+	@Timeout(value = 100, unit = TimeUnit.MILLISECONDS)
+	void failsIfExecutionTimeExceedsFiveSeconds() {
+		// fails if execution time exceeds 100 milliseconds
+	}
+
+}
+// end::user_guide[]

junit-jupiter-api/src/main/java/org/junit/jupiter/api/Timeout.java

@@ -68,6 +68,22 @@
  * overrides {@code junit.jupiter.execution.timeout.testable.method.default}
  * which overrides {@code junit.jupiter.execution.timeout.default}.
  *
+ * <p>Values must be in the following, case-insensitive format:
+ * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
+ * unit may be omitted. Specifying no unit is equivalent to using seconds.
+ *
+ * <table class="plain">
+ * <tr><th> Value         </th><th> Equivalent annotation                             </th></tr>
+ * <tr><td> {@code 42}    </td><td> {@code @Timeout(42)}                              </td></tr>
+ * <tr><td> {@code 42 ns} </td><td> {@code @Timeout(value = 42, unit = NANOSECONDS)}  </td></tr>
+ * <tr><td> {@code 42 μs} </td><td> {@code @Timeout(value = 42, unit = MICROSECONDS)} </td></tr>
+ * <tr><td> {@code 42 ms} </td><td> {@code @Timeout(value = 42, unit = MILLISECONDS)} </td></tr>
+ * <tr><td> {@code 42 s}  </td><td> {@code @Timeout(value = 42, unit = SECONDS)}      </td></tr>
+ * <tr><td> {@code 42 m}  </td><td> {@code @Timeout(value = 42, unit = MINUTES)}      </td></tr>
+ * <tr><td> {@code 42 h}  </td><td> {@code @Timeout(value = 42, unit = HOURS)}        </td></tr>
+ * <tr><td> {@code 42 d}  </td><td> {@code @Timeout(value = 42, unit = DAYS)}         </td></tr>
+ * </table>
+ *
  * @since 5.5
  */
 @Target({ ElementType.TYPE, ElementType.METHOD })

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/Constants.java

@@ -198,7 +198,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -232,7 +232,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -268,7 +268,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -304,7 +304,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -340,7 +340,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -374,7 +374,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -409,7 +409,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -444,7 +444,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -479,7 +479,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *
@@ -514,7 +514,7 @@ public final class Constants {
 	 *
 	 * <h3>Supported Values</h3>
 	 *
-	 * <p>Value must be in the following, case-insensitive format:
+	 * <p>Values must be in the following, case-insensitive format:
 	 * {@code <number> [ns|μs|ms|s|m|h|d]}. The space between the number and the
 	 * unit may be omitted. Specifying no unit is equivalent to using seconds.
 	 *