[CI] TransformContinuousIT.testContinousEvents fails #67887
Description
Build scan:
https://gradle-enterprise.elastic.co/s/d3vdkpkvqmslc
Repro line:
./gradlew ':x-pack:plugin:transform:qa:multi-node-tests:javaRestTest' --tests "org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents" -Dtests.seed=4E7E45250C58F136 -Dtests.security.manager=true -Dtests.locale=fi-FI -Dtests.timezone=Africa/Accra -Druntime.java=11Reproduces locally?:
No
Applicable branches:
master
Started happening on Jan 20th.
Failure excerpt:
org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT > testContinousEvents FAILED
java.lang.AssertionError: transform [continuous-histogram-pivot-test] does not progress, state: STARTED, reason: null
Expected: a value greater than <2021-01-25T07:40:00.506764Z>
but: <2021-01-25T07:39:59.383Z> was less than <2021-01-25T07:40:00.506764Z>
at __randomizedtesting.SeedInfo.seed([4E7E45250C58F136:7548343E86337360]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.lambda$waitUntilTransformsProcessedNewData$3(TransformContinuousIT.java:506)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:955)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.waitUntilTransformsProcessedNewData(TransformContinuousIT.java:504)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents(TransformContinuousIT.java:276)
REPRODUCE WITH: ./gradlew ':x-pack:plugin:transform:qa:multi-node-tests:javaRestTest' --tests "org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents" -Dtests.seed=4E7E45250C58F136 -Dtests.security.manager=true -Dtests.locale=fi-FI -Dtests.timezone=Africa/Accra -Druntime.java=11
Suite: Test class org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT
1> [2021-01-25T07:36:35,379][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] before test
1> [2021-01-25T07:36:35,601][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] initializing REST clients against [http://[::1]:46643, http://127.0.0.1:36227, http://[::1]:37723, http://127.0.0.1:41451, http://[::1]:43801, http://127.0.0.1:40545]
1> [2021-01-25T07:36:36,840][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":0}}]}
1> [2021-01-25T07:36:37,318][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] Creating source index with: {"settings":{"index":{"number_of_shards":1,"codec":"best_compression"}},"mappings":{"properties":{"timestamp":{"type":"date_nanos","format":"strict_date_optional_time_nanos"},"event":{"type":"keyword"},"metric":{"type":"unsigned_long"},"location":{"type":"geo_point"},"run":{"type":"integer"},"metric-timestamp":{"type":"date_nanos"},"some-timestamp":{"type":"date_nanos"}},"runtime":{"metric-rt-2x":{"type":"long","script":{"source":"if (params._source.metric != null) {emit(params._source.metric * 2)}"}},"event-upper":{"type":"keyword","script":{"source":"if (params._source.event != null) {emit(params._source.event.toUpperCase())}"}},"timestamp-at-runtime":{"type":"date","script":{"source":"emit(parse(params._source.get('timestamp')))"}},"metric-timestamp-5m-earlier":{"type":"date","script":{"source":"if (doc['metric-timestamp'].size()!=0) {emit(doc['metric-timestamp'].value.minus(5, ChronoUnit.MINUTES).toInstant().toEpochMilli())}"}},"some-timestamp-10m-earlier":{"type":"date","script":{"source":"if (doc['some-timestamp'].size()!=0) {emit(doc['some-timestamp'].value.minus(10, ChronoUnit.MINUTES).toInstant().toEpochMilli())}"}},"metric":{"type":"long","script":{"source":"if (params._source.metric != null) {emit(params._source.metric * 3)}"}}}}}
1> [2021-01-25T07:36:38,433][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-terms-pivot-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-terms-pivot-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"pivot":{"group_by":{"event":{"terms":{"field":"event","missing_bucket":true}}},"aggregations":{"run.max":{"max":{"field":"run"}},"count":{"value_count":{"field":"run"}},"time.max":{"max":{"field":"timestamp"}},"metric.avg":{"avg":{"field":"metric-rt-2x"}}}},"settings":{"max_page_search_size":10}}
1> [2021-01-25T07:36:39,298][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-terms-on-date-pivot-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-terms-on-date-pivot-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"pivot":{"group_by":{"some-timestamp":{"terms":{"field":"some-timestamp-10m-earlier"}}},"aggregations":{"run.max":{"max":{"field":"run"}},"count":{"value_count":{"field":"run"}},"time.max":{"max":{"field":"timestamp"}},"metric.avg":{"avg":{"field":"metric-rt-2x"}}}},"settings":{"max_page_search_size":10}}
1> [2021-01-25T07:36:39,614][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-date-histogram-pivot-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-date-histogram-pivot-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"pivot":{"group_by":{"second":{"date_histogram":{"field":"timestamp","missing_bucket":true,"fixed_interval":"1s"}}},"aggregations":{"run.max":{"max":{"field":"run"}},"count":{"value_count":{"field":"run"}},"time.max":{"max":{"field":"timestamp"}}}},"settings":{"max_page_search_size":10}}
1> [2021-01-25T07:36:39,748][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-date-histogram-pivot-other-timefield-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-date-histogram-pivot-other-timefield-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"pivot":{"group_by":{"event":{"terms":{"field":"event-upper"}},"second":{"date_histogram":{"field":"metric-timestamp","fixed_interval":"1s"}}},"aggregations":{"run.max":{"max":{"field":"run"}},"count":{"value_count":{"field":"run"}},"time.max":{"max":{"field":"timestamp"}}}},"settings":{"max_page_search_size":10,"dates_as_epoch_millis":true}}
1> [2021-01-25T07:36:39,934][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-histogram-pivot-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-histogram-pivot-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"pivot":{"group_by":{"metric":{"histogram":{"field":"metric-rt-2x","interval":50.0}}},"aggregations":{"run.max":{"max":{"field":"run"}},"count":{"value_count":{"field":"run"}},"time.max":{"max":{"field":"timestamp"}}}},"settings":{"max_page_search_size":10}}
1> [2021-01-25T07:36:40,051][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putTransform: {"id":"continuous-latest-test","source":{"index":["test-transform-continuous-events"]},"dest":{"index":"continuous-latest-test","pipeline":"transform-ingest"},"frequency":"1s","sync":{"time":{"field":"timestamp","delay":"1s"}},"latest":{"unique_key":["event"],"sort":"timestamp"},"settings":{"max_page_search_size":10}}
1> [2021-01-25T07:36:40,161][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":0}}]}
1> [2021-01-25T07:36:49,717][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:36:50,341][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:36:50,775][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:36:51,515][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:36:52,166][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:36:52,774][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:36:53,297][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:36:54.296824Z (takes into account the delay: 1s) iteration: 0
1> [2021-01-25T07:36:56,013][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:36:56,151][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:36:56,294][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:36:56,406][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:36:56,518][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:36:56,632][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:36:56,826][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://127.0.0.1:36227/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:36:57,659][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":1}}]}
1> [2021-01-25T07:37:04,267][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:04,405][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:04,544][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:04,685][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:04,854][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:05,044][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:37:05,224][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:37:06.224574Z (takes into account the delay: 1s) iteration: 1
1> [2021-01-25T07:37:11,253][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:11,379][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:11,471][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:11,579][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:11,676][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:11,784][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:37:11,934][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://[::1]:46643/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:37:12,781][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":2}}]}
1> [2021-01-25T07:37:15,710][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:15,858][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:16,042][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:16,204][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:16,356][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:16,504][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:37:16,658][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:37:17.658275Z (takes into account the delay: 1s) iteration: 2
1> [2021-01-25T07:37:26,820][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:26,917][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:27,017][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:27,112][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:27,211][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:27,300][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:37:27,474][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://[::1]:46643/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:37:28,366][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":3}}]}
1> [2021-01-25T07:37:34,367][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:34,487][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:34,632][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:34,764][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:34,910][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:35,060][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:37:35,195][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:37:36.194908Z (takes into account the delay: 1s) iteration: 3
1> [2021-01-25T07:37:48,116][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:48,212][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:48,301][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:48,396][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:48,492][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:48,578][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:37:48,752][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://[::1]:46643/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:37:49,866][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":4}}]}
1> [2021-01-25T07:37:54,843][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:37:54,995][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:37:55,161][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:37:55,320][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:37:55,488][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:37:55,643][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:37:55,801][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:37:56.801269Z (takes into account the delay: 1s) iteration: 4
1> [2021-01-25T07:38:18,818][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:38:18,917][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:38:19,008][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:38:19,100][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:38:19,205][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:38:19,294][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:38:19,462][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://127.0.0.1:41451/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:38:20,882][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":5}}]}
1> [2021-01-25T07:38:26,736][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:38:26,874][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:38:27,010][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:38:27,134][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:38:27,266][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:38:27,393][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:38:27,505][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:38:28.505661Z (takes into account the delay: 1s) iteration: 5
1> [2021-01-25T07:38:50,647][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:38:50,747][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:38:50,836][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:38:50,964][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:38:51,066][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:38:51,168][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:38:51,336][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://127.0.0.1:36227/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:38:52,898][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":6}}]}
1> [2021-01-25T07:38:55,936][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:38:56,048][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:38:56,165][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:38:56,296][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:38:56,425][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:38:56,558][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:38:56,680][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:38:57.680161Z (takes into account the delay: 1s) iteration: 6
1> [2021-01-25T07:39:19,153][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:39:19,244][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:39:19,325][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:39:19,415][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:39:19,506][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:39:19,597][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:39:19,744][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://[::1]:46643/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:39:21,388][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":7}}]}
1> [2021-01-25T07:39:27,382][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:39:27,498][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:39:27,624][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:39:27,746][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:39:27,862][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:39:28,008][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:39:28,129][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:39:29.12901Z (takes into account the delay: 1s) iteration: 7
1> [2021-01-25T07:39:52,798][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-pivot-test
1> [2021-01-25T07:39:52,885][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:39:52,970][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:39:53,054][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:39:53,146][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:39:53,231][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] stopTransform: continuous-latest-test
1> [2021-01-25T07:39:53,369][WARN ][o.e.c.RestClient ] [testContinousEvents] request [POST http://[::1]:46643/_refresh?expand_wildcards=open%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:39:55,290][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] putPipeline: {"processors":[{"set":{"field":"run_ingest","value":8}}]}
1> [2021-01-25T07:39:58,763][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-pivot-test
1> [2021-01-25T07:39:58,882][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:39:59,004][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:39:59,125][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:39:59,249][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:39:59,386][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] startTransform: continuous-latest-test
1> [2021-01-25T07:39:59,506][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] wait until transform reaches timestamp_millis: 2021-01-25T07:40:00.506764Z (takes into account the delay: 1s) iteration: 8
1> [2021-01-25T07:40:22,708][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-date-histogram-pivot-other-timefield-test
1> [2021-01-25T07:40:22,909][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-date-histogram-pivot-test
1> [2021-01-25T07:40:23,058][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-histogram-pivot-test
1> [2021-01-25T07:40:23,213][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-latest-test
1> [2021-01-25T07:40:23,356][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-terms-on-date-pivot-test
1> [2021-01-25T07:40:23,493][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deleteTransform: continuous-terms-pivot-test
1> [2021-01-25T07:40:23,645][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] deletePipeline: transform-ingest
1> [2021-01-25T07:40:23,983][WARN ][o.e.c.RestClient ] [testContinousEvents] request [DELETE http://[::1]:37723/*,-.ds-ilm-history-*?expand_wildcards=open%2Cclosed%2Chidden] returned 1 warnings: [299 Elasticsearch-8.0.0-SNAPSHOT-63c85ff3e846a13ae1294e99fc8ad63b9dd72d38 "this request accesses system indices: [.transform-internal-005], but in a future major version, direct access to system indices will be prevented by default"]
1> [2021-01-25T07:40:24,335][INFO ][o.e.x.t.i.c.TransformContinuousIT] [testContinousEvents] after test
2> REPRODUCE WITH: ./gradlew ':x-pack:plugin:transform:qa:multi-node-tests:javaRestTest' --tests "org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents" -Dtests.seed=4E7E45250C58F136 -Dtests.security.manager=true -Dtests.locale=fi-FI -Dtests.timezone=Africa/Accra -Druntime.java=11
2> java.lang.AssertionError: transform [continuous-histogram-pivot-test] does not progress, state: STARTED, reason: null
Expected: a value greater than <2021-01-25T07:40:00.506764Z>
but: <2021-01-25T07:39:59.383Z> was less than <2021-01-25T07:40:00.506764Z>
at __randomizedtesting.SeedInfo.seed([4E7E45250C58F136:7548343E86337360]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.lambda$waitUntilTransformsProcessedNewData$3(TransformContinuousIT.java:506)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:955)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.waitUntilTransformsProcessedNewData(TransformContinuousIT.java:504)
at org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents(TransformContinuousIT.java:276)
2> NOTE: leaving temporary files on disk at: /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob-unix-compatibility/os/opensuse-15-1&&immutable/x-pack/plugin/transform/qa/multi-node-tests/build/testrun/javaRestTest/temp/org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT_4E7E45250C58F136-001
2> NOTE: test params are: codec=Asserting(Lucene87): {}, docValues:{}, maxPointsInLeafNode=692, maxMBSortInHeap=7.427266613072976, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=fi-FI, timezone=Africa/Accra
2> NOTE: Linux 4.12.14-lp151.28.91-default amd64/Oracle Corporation 11.0.2 (64-bit)/cpus=32,threads=1,free=477713648,total=536870912
2> NOTE: All tests run in this JVM: [DateHistogramGroupByIT, DateHistogramGroupByOtherTimeFieldIT, HistogramGroupByIT, LatestContinuousIT, TermsGroupByIT, TermsOnDateGroupByIT, TransformContinuousIT]
Tests with failures:
- org.elasticsearch.xpack.transform.integration.continuous.TransformContinuousIT.testContinousEvents
12 tests completed, 1 failed