Skip to content

Conversation

@jkakavas
Copy link
Contributor

Backports the following commits to 7.x:

Reverts 71968c9. The underlying
issue which was related to the bouncycastle security provider
permissions was resolved in
elastic/infra#14995

resolves: elastic#39981
@jkakavas
Copy link
Contributor Author

@elasticmachine test this please

@jkakavas
Copy link
Contributor Author

jkakavas commented Oct 14, 2019

BWC tests fail because for 6.8.4 nodes we get

org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT > test {p0=search/180_locale_dependent_mapping/Test Index and Search locale dependent mappings / dates} FAILED
--
java.lang.AssertionError: Failure at [search/180_locale_dependent_mapping:6]: got unexpected warning header [
299 Elasticsearch-6.8.4-SNAPSHOT-75f6eb7 "'y' year should be replaced with 'u'. Use 'y' for year-of-era.; 'Z' time zone offset/id fails when parsing 'Z' for Zulu timezone. Consider using 'X'. Prefix your date format with '8' to use the new specifier."
]
 
Caused by:
java.lang.AssertionError: got unexpected warning header [
299 Elasticsearch-6.8.4-SNAPSHOT-75f6eb7 "'y' year should be replaced with 'u'. Use 'y' for year-of-era.; 'Z' time zone offset/id fails when parsing 'Z' for Zulu timezone. Consider using 'X'. Prefix your date format with '8' to use the new specifier."
]
REPRODUCE WITH: ./gradlew ':qa:mixed-cluster:v6.8.4#mixedClusterTest' --tests "org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=search/180_locale_dependent_mapping/Test Index and Search locale dependent mappings / dates}" -Dtests.seed=F4D9B06BB0B3F0CD -Dtests.security.manager=true -Dtests.locale=vi-VN -Dtests.timezone=Europe/Kiev -Dcompiler.java=12 -Druntime.java=8

@pgomulka any suggestions whether we should change the pattern in 7.x or simply disregard the warning ?

@pgomulka
Copy link
Contributor

pgomulka commented Oct 15, 2019

If possible it would be great to migrate it to use recommended pattern. This will make future changes around dates easier.

I am planning to refactor our tests to all use recommended patterns, but if you can fix this one that would help me a lot

@jkakavas
Copy link
Contributor Author

@pgomulka do you have any idea from the top of your head for how this should change ? Warning message said

'y' year should be replaced with 'u'

but neither

format: "E, d MMM uuuu HH:mm:ss Z"

nor

format: "E, d MMM u HH:mm:ss Z"

can be parsed in 6.8 nodes .

@pgomulka
Copy link
Contributor

@jkakavas in 6.8 the warning is created when new index with old joda style pattern is created.
To avoid that, in 6.8 new indices should prefix their patterns with 8 and use new java.time patterns

format: "8E, d MMM uuuu HH:mm:ss Z"

This is parsable in both 6.8 and 7+

@pgomulka
Copy link
Contributor

however.. the behaviour of parsing is depending on jdk version. I noticed that locally on jdk 12 it was failing for me. Then I found this: https://stackoverflow.com/questions/46244724/jdk-dateformatter-parsing-dayofweek-in-german-locale-java8-vs-java9/46245412#46245412
since jdk9 the locale data has changed, and it would expect Mi. instead of just Mi same for Dez. instead of just Dez
It should not be a problem when running in prod as we provide jdk flag -Djava.locale.providers=COMPAT which makes the behaviour to be compatible with jdk8

Somehow this jdk option is not provided for this test?

@jkakavas
Copy link
Contributor Author

jkakavas commented Oct 16, 2019

AFAICT, all nodes run with java 8

12:41:28  [7.4.1]   Runtime JDK Version   : 1.8 (Oracle Corporation 1.8.0_221 [Java HotSpot(TM) 64-Bit Server VM 25.221-b11])
12:41:28  [7.4.1]   Runtime java.home     : /var/lib/jenkins/.java/java8
...
12:41:40  [6.8.4]   Runtime JDK Version   : 1.8 (Oracle Corporation 1.8.0_221 [Java HotSpot(TM) 64-Bit Server VM 25.221-b11])
12:41:40  [6.8.4]   Runtime java.home     : /var/lib/jenkins/.java/java8
...
12:42:38  [7.4.1]   Runtime JDK Version   : 1.8 (Oracle Corporation 1.8.0_221 [Java HotSpot(TM) 64-Bit Server VM 25.221-b11])
12:42:38  [7.4.1]   Runtime java.home     : /var/lib/jenkins/.java/java8
...
12:43:33  [6.8.4]   Runtime JDK Version   : 1.8 (Oracle Corporation 1.8.0_221 [Java HotSpot(TM) 64-Bit Server VM 25.221-b11])
12:43:33  [6.8.4]   Runtime java.home     : /var/lib/jenkins/.java/java8

@jkakavas
Copy link
Contributor Author

Caused by: java.lang.IllegalStateException: DocumentMapper serialization result is different from source. 
--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]
--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]
	at org.elasticsearch.index.mapper.MapperService.assertSerialization(MapperService.java:617)

@pgomulka
Copy link
Contributor

The main problem is to create a mapping with a locale and a format prefixed with 8. There is a bugfix for this #48139
Once it is merged, this failure should be fixed.

pgomulka added a commit that referenced this pull request Oct 17, 2019
It happens when a locale is specified on a mapping that a new
JavaDateFormatter is created with a new mapping and already prefixed
with 8 format is prefixed again. This should change avoids this
behavior.
relates  #47983
@pgomulka
Copy link
Contributor

ok to test

@jkakavas
Copy link
Contributor Author

@elasticmachine update branch

@jkakavas
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/default-distro

@jkakavas
Copy link
Contributor Author

@elasticmachine test this please again so that I can sleep well at night

@jkakavas
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/bwc

@jkakavas
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/default-distro

@pgomulka
Copy link
Contributor

@elasticmachine update branch

@pgomulka
Copy link
Contributor

@elasticmachine run elasticsearch-ci/bwc

1 similar comment
@pgomulka
Copy link
Contributor

@elasticmachine run elasticsearch-ci/bwc

@jkakavas
Copy link
Contributor Author

@elasticmachine update branch

@pgomulka
Copy link
Contributor

@elasticmachine update branch

@jkakavas
Copy link
Contributor Author

^^ hmm... I would expect more CI jobs

@elasticmachine test this ( all of this (no really all the jobs please ) ) please

@pgomulka pgomulka added the :Core/Infra/Core Core issues without another label label Nov 27, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Core)

@pgomulka
Copy link
Contributor

@jkakavas I just realised that I have re-enabled this test in #48703
It is interesting though how the CI behaved on empty PR :)

@dliappis
Copy link
Contributor

@pgomulka

It is interesting though how the CI behaved on empty PR :)

There are a lot of CI failures since the tests got unmuted.
Example is: https://gradle-enterprise.elastic.co/s/m64s5k5k3pbn6/tests/qiq773hxdcg7e-2pri752okzdby

Example failure:

expected [2xx] status code but api [indices.create] returned [500 Internal Server Error] [{"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[v6.8.3-2][127.0.0.1:36768][indices:admin/create]","stack_trace":"[[v6.8.3-2][127.0.0.1:36768][indices:admin/create]]; nested: RemoteTransportException[[v6.8.3-2][127.0.0.1:36768][indices:admin/create]]; nested: IllegalStateException[DocumentMapper serialization result is different from source. \n--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]];\n\tat org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:644)\n\tat org.elasticsearch.ElasticsearchException.generateFailureXContent(ElasticsearchException.java:572)\n\tat org.elasticsearch.rest.BytesRestResponse.build(BytesRestResponse.java:138)\n\tat org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:96)\n\tat org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:91)\n\tat org.elasticsearch.rest.action.RestActionListener.onFailure(RestActionListener.java:58)\n\tat org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:79)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$1.handleException(TransportMasterNodeAction.java:190)\n\tat org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1119)\n\tat org.elasticsearch.transport.InboundHandler.lambda$handleException$2(InboundHandler.java:243)\n\tat org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225)\n\tat org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:241)\n\tat org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:233)\n\tat org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:136)\n\tat org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:102)\n\tat org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:673)\n\tat org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:62)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)\n\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)\n\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:600)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:554)\n\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)\n\tat io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)\n\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: RemoteTransportException[[v6.8.3-2][127.0.0.1:36768][indices:admin/create]]; nested: IllegalStateException[DocumentMapper serialization result is different from source. \n--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]];\nCaused by: java.lang.IllegalStateException: DocumentMapper serialization result is different from source. \n--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n\tat org.elasticsearch.index.mapper.MapperService.assertSerialization(MapperService.java:617)\n\tat java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)\n\tat java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)\n\tat java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)\n\tat java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)\n\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)\n\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)\n\tat java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)\n\tat java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)\n\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat java.util.stream.ReferencePipeline.allMatch(ReferencePipeline.java:454)\n\tat org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:592)\n\tat org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:403)\n\tat org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:330)\n\tat org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:481)\n\tat org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:643)\n\tat org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:270)\n\tat org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:200)\n\tat org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:135)\n\tat org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)\n\tat org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n"}],"type":"illegal_state_exception","reason":"DocumentMapper serialization result is different from source. \n--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]","stack_trace":"java.lang.IllegalStateException: DocumentMapper serialization result is different from source. \n--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]\n\tat org.elasticsearch.index.mapper.MapperService.assertSerialization(MapperService.java:617)\n\tat java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)\n\tat java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)\n\tat java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)\n\tat java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)\n\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)\n\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)\n\tat java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)\n\tat java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)\n\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat java.util.stream.ReferencePipeline.allMatch(ReferencePipeline.java:454)\n\tat org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:592)\n\tat org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:403)\n\tat org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:330)\n\tat org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$IndexCreationTask.execute(MetaDataCreateIndexService.java:481)\n\tat org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:643)\n\tat org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:270)\n\tat org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:200)\n\tat org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:135)\n\tat org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)\n\tat org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)\n\tat org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n"},"status":500}]Open stacktrace
[2019-11-29T08:03:40,087][INFO ][o.e.b.MixedClusterClientYamlTestSuiteIT] [test] [p0=search/180_locale_dependent_mapping/Test Index and Search locale dependent mappings / dates] before test
11 29, 2019 8:03:40 午前 org.elasticsearch.client.RestClient logResponse
警告: request [PUT http://[::1]:34553/test_index?include_type_name=false&error_trace=true] returned 1 warnings: [299 Elasticsearch-7.6.0-SNAPSHOT-90e9d61f2b75aa2962685175ea1bd92b8bb7c223 "[types removal] Using include_type_name in create index requests is deprecated. The parameter will be removed in the next major version."]
[2019-11-29T08:03:40,108][INFO ][o.e.b.MixedClusterClientYamlTestSuiteIT] [test] Stash dump on test failure [{
  "stash" : {
    "body" : {
      "error" : {
        "root_cause" : [
          {
            "type" : "remote_transport_exception",
            "reason" : "[v6.8.3-2][127.0.0.1:36768][indices:admin/create]",
            "stack_trace" : "[[v6.8.3-2][127.0.0.1:36768][indices:admin/create]]; nested: RemoteTransportException[[v6.8.3-2][127.0.0.1:36768][indices:admin/create]]; nested: IllegalStateException[DocumentMapper serialization result is different from source. 
--> Source [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"88E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]
--> Result [{\"_doc\":{\"properties\":{\"date_field\":{\"type\":\"date\",\"format\":\"888E, d MMM uuuu HH:mm:ss Z\",\"locale\":\"de\"}}}}]];

Are you ok if I mute again until this gets looked at again?

@jkakavas
Copy link
Contributor Author

Are you ok if I mute again until this gets looked at again?

@dliappis let's mute this

@dliappis
Copy link
Contributor

@jkakavas ACK

dliappis added a commit to dliappis/elasticsearch that referenced this pull request Nov 29, 2019
@dliappis
Copy link
Contributor

Raised #49721 to mute the test

dliappis added a commit that referenced this pull request Nov 29, 2019
@pgomulka pgomulka closed this Dec 11, 2019
@pgomulka
Copy link
Contributor

closing as the test was fixed in #48703
and skip versions set in #50028

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport :Core/Infra/Core Core issues without another label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants