lifecycle_replace_after_pause is flaky

[rjrjr]

Hard to reproduce, but if I run it hundreds of times against a Pixel_2_API_21 AVD, every now and then it fails with:

com.squareup.workflow1.ui.backstack.test.BackstackContainerTest3 > lifecycle_replace_after_pause21[Pixel_2_API_21(AVD) - 5.0.2] FAILED 
        java.lang.RuntimeException: missing (1)                 : LeafView next ON_START

java.lang.RuntimeException: missing (1)                 : LeafView next ON_START---expected to contain
at least: [LeafView initial ON_PAUSE, activity onPause, initial onDetach viewState=, next onAttach viewState=, LeafView next ON_CREATE, LeafView next ON_START, LeafView initial ON_STOP, LeafView initial ON_DESTROY]but was                     : [LeafView initial ON_PAUSE, activity onPause, LeafView initial ON_STOP, initial onSave viewState=, next onViewCreated viewState=, next onShowRendering viewState=, next onShowRendering viewState=, initial onDetach viewState=, next onAttach viewState=, LeafView next ON_CREATE, LeafView initial ON_DESTROY]
  at androidx.test.runner.MonitoringInstrumentation.runOnMainSync(MonitoringInstrumentation.java:450)
  at androidx.test.core.app.ActivityScenario.onActivity(ActivityScenario.java:673)
  at com.squareup.workflow1.ui.backstack.test.BackstackContainerTest.lifecycle_replace_after_pause(BackstackContainerTest.kt:364)
  at java.lang.reflect.Method.invoke(Native Method)
  at java.lang.reflect.Method.invoke(Method.java:372)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
  at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
  at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
  at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
  at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
  at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
  at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
  at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
  at org.junit.runners.Suite.runChild(Suite.java:128)
  at org.junit.runners.Suite.runChild(Suite.java:27)
  at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
  at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
  at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
  at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
  at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:395)
  at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1837)
Caused by: missing (1)                 : LeafView next ON_START---expected to contain
at least: [LeafView initial ON_PAUSE, activity onPause, initial onDetach viewState=, next onAttach viewState=, LeafView next ON_CREATE, LeafView next ON_START, LeafView initial ON_STOP, LeafView initial ON_DESTROY]but was                     : [LeafView initial ON_PAUSE, activity onPause, LeafView initial ON_STOP, initial onSave viewState=, next onViewCreated viewState=, next onShowRendering viewState=, next onShowRendering viewState=, initial onDetach viewState=, next onAttach viewState=, LeafView next ON_CREATE, LeafView initial ON_DESTROY]
  at com.squareup.workflow1.ui.backstack.test.BackstackContainerTest.lifecycle_replace_after_pause$lambda-37(BackstackContainerTest.kt:365)
  at com.squareup.workflow1.ui.backstack.test.BackstackContainerTest.$r8$lambda$mCsgI3ooA1pqbmDyU_PosZos40U(BackstackContainerTest.kt)
  at com.squareup.workflow1.ui.backstack.test.BackstackContainerTest$$ExternalSyntheticLambda35.perform(Unknown Source)
  at androidx.test.core.app.ActivityScenario.lambda$onActivity$2$ActivityScenario(ActivityScenario.java:660)
  at androidx.test.core.app.ActivityScenario$$Lambda$4.run(Unknown Source)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
  at java.util.concurrent.FutureTask.run(FutureTask.java:237)
  at android.app.Instrumentation$SyncRunnable.run(Instrumentation.java:1855)
  at android.os.Handler.handleCallback(Handler.java:739)
  at android.os.Handler.dispatchMessage(Handler.java:95)
  at android.os.Looper.loop(Looper.java:135)
  at android.app.ActivityThread.main(ActivityThread.java:5221)
  at [[Reflective call: 2 frames collapsed (https://goo.gl/aH3UyP)]].(:0)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:899)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:694)

[rjrjr]

My immediate suspicion is around the WeakReference<View> in WorkflowLifecycleOwner.

[rjrjr]

Dropping the WeakReference is satisfying, but doesn't fix the flake.

[rjrjr]

It's always next ON_START, never the first one. What happens on other SDK versions?

[rjrjr]

Cannot reproduce with SDK 26.

Tomorrow me gets to understand if the problem really is Android skipping a lifecycle event (I believe it); and if that might actually break something, or if I can instead just tweak the test.

[rjrjr]

Really hard to reproduce. I made 8 copies of BackstackContainerTest, each containing only 22 copies of lifecycle_replace_after_pause. Out of those 176 runs, I'll generally see 3 failures. Call it a 1.5% flake rate.

[rjrjr]

So really this is about the order in which one activity is started after another stops, and whether the second one moves to START state before the first one is destroyed and the test ends?

[rjrjr]

Be interesting to put a third, fourth RESUMED / STARTED traversal in there -- see if this is always about the final one or what.