[CI] search/180_locale_dependent_mapping/Test Index and Search locale dependent mappings / dates

[andyb-elastic]

Failed in a couple periodic builds today, looks like they're all on FIPS but that may just be coincidence

• https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+matrix-java-periodic/ES_BUILD_JAVA=openjdk12,ES_RUNTIME_JAVA=java8fips,nodes=immutable&&linux&&docker/294/console
• https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+matrix-java-periodic/ES_BUILD_JAVA=java11,ES_RUNTIME_JAVA=java8fips,nodes=immutable&&linux&&docker/294/console
• https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=java11,ES_RUNTIME_JAVA=java8fips,nodes=immutable&&linux&&docker/69/console
• https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=openjdk12,ES_RUNTIME_JAVA=java8fips,nodes=immutable&&linux&&docker/69/console

Here's a log from one
build-periodic-7.x-69.txt.zip

REPRODUCE WITH: ./gradlew :distribution:archives:integ-test-zip:integTestRunner -Dtests.seed=DE4EDE4CD5F93BE9 -Dtests.class=org.elasticsearch.test.rest.IntegTestZipClientYamlTestSuiteIT -Dtests.method="test {p0=search/180_locale_dependent_mapping/Test Index and Search locale dependent mappings / dates}" -Dtests.security.manager=true -Dtests.locale=es-PY -Dtests.timezone=Asia/Bahrain -Dcompiler.java=12 -Druntime.java=8FIPS -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStorePassword=password

  1>             "type" : "parse_exception",
  1>             "reason" : "failed to parse date field [Di, 05 Dez 2000 02:55:00 -0800] with format [E, d MMM yyyy HH:mm:ss Z]: [Text 'Di, 05 Dez 2000 02:55:00 -0800' could not be parsed at index 0]",
  1>             "stack_trace" : "ElasticsearchParseException[failed to parse date field [Di, 05 Dez 2000 02:55:00 -0800] with format [E, d MMM yyyy HH:mm:ss Z]: [Text 'Di, 05 Dez 2000 02:55:00 -0800' could not be parsed at index 0]]; nested: DateTimeParseException[Text 'Di, 05 Dez 2000 02:55:00 -0800' could not be parsed at index 0];
  1>    at org.elasticsearch.common.time.JavaDateMathParser.parseDateTime(JavaDateMathParser.java:233)
  1>    at org.elasticsearch.common.time.JavaDateMathParser.parse(JavaDateMathParser.java:75)
  1>    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToLong(DateFieldMapper.java:397)
  1>    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:411)
  1>    at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:447)
  1>    at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:464)
  1>    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:265)
  1>    at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:962)
  1>    at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:82)
  1>    at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68)
  1>    at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:51)
  1>    at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:271)
  1>    at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:261)
  1>    at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68)
  1>    at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:659)
  1>    at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:638)
  1>    at org.elasticsearch.search.SearchService.createContext(SearchService.java:601)
  1>    at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:582)
  1>    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:385)
  1>    at org.elasticsearch.search.SearchService.access$100(SearchService.java:123)
  1>    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:357)
  1>    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:353)
  1>    at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1068)
  1>    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  1>    at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:43)
  1>    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:747)
  1>    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
  1>    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1>    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1>    at java.lang.Thread.run(Thread.java:748)
  1> Caused by: java.time.format.DateTimeParseException: Text 'Di, 05 Dez 2000 02:55:00 -0800' could not be parsed at index 0
  1>    at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:1949)
  1>    at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1777)
  1>    at org.elasticsearch.common.time.JavaDateMathParser.parseDateTime(JavaDateMathParser.java:221)
  1>    ... 29 more

[elasticmachine]

Pinging @elastic/es-search

[jkakavas]

failed to parse date field

and FIPS 140 JVMs are unfortunately not a coincidence

As we figured out in #39855, the integTestClusters started to run with FIPS JVMs recently and this is why this came up only now. I will resolve this in a similar way as in #39052

[elasticmachine]

Pinging @elastic/es-security

[jkakavas]

It looks like there might be an issue with the E, d MMM yyyy HH:mm:ss Z format and the Java 8, judging from
https://github.com/elastic/elasticsearch/blob/daa2ec8a605d385a65b9ab3e89d016b3fd0dffe2/server/src/test/java/org/elasticsearch/index/mapper/DateFieldMapperTests.java#L223:L224 and as such this is not fips specific, but rather java 8 specific. @spinscale can maybe share some insights about E, d MMM yyyy HH:mm:ss Z

[jkakavas]

This doesn't seem to be java 8 specific ( at least not entirely ) since it only fails with a java 8 fips runtime : https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-7d,mode:quick,to:now))&_a=(columns:!(build.url),index:b646ed00-7efc-11e8-bf69-63c8ef516157,interval:auto,query:(language:lucene,query:%22180_locale_dependent_mapping%22),sort:!(process.time-start,desc))

We don't have a way to mute yaml tests based on JVM version so I will mute this entirely so the rests of the tests can run in FIPS, until we can figure out what causes the parsing error in FIPS

[tvernum]

Are your Java 8 FIPS and Java 8 Standard environments on the same JDK update?
I recently had some timezone tests that worked on jdk1.8.0_201 but fail on jdk1.8.0_111

[jkakavas]

Good point @tvernum . My standard is 1.8.0_191 while I had based FIPS on 1.8.0_171. But I guess in CI both are built by the same 8 version so these tests would be failing on java8 also.. I'll update my local one and investigate further nevertheless

[jkakavas]

This still fails with a FIPS enabled JVM based on jdk1.8.0_201. Investigation ongoing