Provide a way to specify my own test method runner

[MatousJobanek]

Overview

I'm part of Arquillian team and I've been trying to implement either Arquillian Engine or Arquilian Extension for JUnit 5. Unfortunately, nothing is feasible for now. The Engine is a bigger issue. In case of extension, there is missing only one functionality.

Specific problem

One of Arquillian modes is running in-container tests - executing test methods/classes inside of a container (as part of a deployment). Unfortunately, currently there is no way to tell JUnit 5 where the test should be executed or to provide my own runner that would execute the test method either on a client or in a container. I can only skip the test methods or let them run locally.

Possible solutions

Custom ExecutableInvoker

The easiest solution would be having an SPI (eg. using service loader) that would load inside of the TestMethodTestDescriptor a custom implementation of the ExecutableInvoker class (if none is provided, then the default would be used) and execute the method using the provided invoker.

Custom runner for test descriptors

The previous solution is easy to do, but has one side effect - the method callbacks would be executed on both sides - on a client and inside of the container (as I would need to simulate everything inside of the container again). An advanced solution would be having a possibility to provide an invoker for every TestDescriptor separately.

For more information about my research see this issue: arquillian/arquillian-core#137

Related Issues

• Introduce extension API for executing test in user-defined thread #157

[marcphilipp]

We're going to provide a way to execute a test (for #20 and #60) using a custom "ExecutorService"-like interface. The "Executable" we're planning to hand to this "ExecutorService" would, when executed, fire events to EngineExecutionListeners. I'm not sure if that would suit your needs or whether you need something more fine-grained.

Is this request different from #1145?

[marcphilipp]

I don't think providing an ExecutableInvoker would be sufficient. All extension code should also be executed inside the Arquillian container, shouldn't it? In addition, the class-level @BeforeAll/@AfterAll methods would have to be executed there, too, right?

[MatousJobanek]

Hi,
thanks for your reply and for the links - yeah it is mainly related to the #20 and #1145 - to intercept/replace the execution of the particular phases - whole class or method blocks.

ExecutableInvoker is the simplest solution and would be partially sufficient (there are some corner cases) - the execution of the tests inside of the container is invoked by rest calls and then the execution itself is done using JUnit 5 API with a new request:

LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
    .selectors(DiscoverySelectors.selectMethod(testClass.getCanonicalName(), methodName))
    .build();
launcher.execute(request);

so, then I have the whole test lifecycle executed inside of the container again.

But, having the way of providing my own executor for the test method block (including the callbacks) would be the best.

[marcphilipp]

How would this work in detail?

Let's assume we introduced a new ExecutableInvoker interface. I imagine it would look similar to this:

interface ExecutableInvoker extends Extension {
  void invoke(Executable executable, ExtensionContext extensionContext) throws Throwable;
}

What would the Arquillian implementation do in the non-embedded container case? Would it completely ignore the Executable and pass on the unique ID or test class and method from the ExtensionContext to the other JVM? The latter would cause class-level lifecycle callbacks to be executed again, wouldn't it?

[jonpeterson]

I'd like to see something like this so I could write an invoker and stop having to wrap all my tests that call Kotlin Coroutines suspend functions in runBlocking { ... }.

[sbrannen]

This issue is related to #157.

[arjantijms]

This issue is related to #157.

With #157 resolved, is there any progress on this issue now?

[sbrannen]

@MatousJobanek and @arjantijms, does the support introduced in #157 and #2083 meet your needs?