org.junit.rules.Timeout does skip @After execution

[arcuri82]

Hi,
I noticed that in my tests the @after methods were not executed in the case of test timeouts. However, that only happens when the Timeout rule is used. For example, the following test is fine (in the sense After method is called as expected):

public class CheckTimeoutProblem {
    @Before
    public void before(){ System.out.println("Called @Before");}

    @After
    public void after(){System.out.println("Called @After");}

    @Test(timeout = 10)
    public void doFailWithTimeout(){
        while(true){ int x = 0; x++;}
    }

    @Test
    public void emptyTest(){}
}

When running this, I get:

Called @before
Called @after
Called @before
Called @after

which is as expected. However, if I use Timeout Rule like:

public class CheckTimeoutProblem {
    @Rule
    public org.junit.rules.Timeout globalTimeout = new org.junit.rules.Timeout(200);

    @Before
    public void before(){ System.out.println("Called @Before");}

    @After
    public void after(){System.out.println("Called @After");}

    @Test
    public void doFailWithTimeout(){
        while(true){ int x = 0; x++;}
    }

   @Test
    public void emptyTest(){}
}

Called @before
Called @after
Called @before

where the After of the timed out test is not executed :(

[kcooney]

Unfortunately, I don't see a way we can fix this. Rules work "around" the test fixture (including any @Before and @After methods), and specifies a timeout for the entire test fixture. In contrast, the timeout attribute is implemented by the runner (which also implents the @After methods) and it specifies a timeout on the test method, so it can execute the @After methods if the test method times out.

Both mechanisms interrupt the test thread on timeout, so if your test did a blocking operation (like a sleep call or waiting on a semaphore) the @After methods would be called.

[arcuri82]

I see. Thanks for the clarification. If it cannot be fixed, I would suggest at least the following:

1. Update the documentation at https://github.com/junit-team/junit/wiki/Timeout-for-tests , making very clear that the two timeout mechanisms (in @test and in @rule) are very different, where @rule does not guarantee that any @after method will be called.

2. Provide a new mechanism to have a global timeout on tests (instead of being forced to repeat it in each single @test annotation), where @after is guaranteed to be called if a test does timeout.

[kcooney]

I updated the wiki page. Please let me know if the new edits don't clarify the situation.

Note that if the test times out during an operation that is interruptible then @After methods would be called. I don't think that handling infinite loops was the primary goal of the timeout features; I updated the wiki to have more realistic examples.

Having a global timeout outside of a rule is an interesting idea, but I'm not sure how we would make it work for all runners (we might be able to get it to work for runners that extend ParentRunner). I suggest filing a feature request for JUnit Lambda (https://github.com/junit-team/junit-lambda)

[arcuri82]

thanks for updating the documentation

i made a feature request at

junit-team/junit-framework#80

[quiram]

This might be a bit redundant, but I wonder if it's worth mentioning in the docs that this obviously affects also @AfterClass, and not just @After (this is what bit me).