Test Failure org.elasticsearch.xpack.sql.qa.single_node.JdbcResultSetIT.testGettingDateWithoutCalendar

[benwtrent]

Able to reproduce reliably:

./gradlew ':x-pack:plugin:sql:qa:single-node:integTestRunner' \
  -Dtests.seed=2261437312AC94BD \
  -Dtests.class=org.elasticsearch.xpack.sql.qa.single_node.JdbcResultSetIT \
  -Dtests.method="testGettingDateWithoutCalendar" \
  -Dtests.security.manager=true \
  -Dtests.locale=id \
  -Dtests.timezone=Indian/Antananarivo \
  -Dcompiler.java=12 \
  -Druntime.java=13

Failure:

FAILURE 1.91s | JdbcResultSetIT.testGettingDateWithoutCalendar <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: expected:<7167-11-30> but was:<7167-12-01>
   >    at __randomizedtesting.SeedInfo.seed([2261437312AC94BD:1B08EC485054097C]:0)
   >    at org.elasticsearch.xpack.sql.qa.jdbc.ResultSetTestCase.lambda$testGettingDateWithoutCalendar$134(ResultSetTestCase.java:886)
   >    at org.elasticsearch.xpack.sql.qa.jdbc.ResultSetTestCase.doWithQuery(ResultSetTestCase.java:1402)
   >    at org.elasticsearch.xpack.sql.qa.jdbc.ResultSetTestCase.doWithQuery(ResultSetTestCase.java:1390)
   >    at org.elasticsearch.xpack.sql.qa.jdbc.ResultSetTestCase.testGettingDateWithoutCalendar(ResultSetTestCase.java:881)
   >    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   >    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
   >    at java.base/java.lang.Thread.run(Thread.java:830)

Seems specifically related to Java13. I tried without Java 13 and it passed.

[elasticmachine]

Pinging @elastic/es-search (:Search/SQL)

[droberts195]

This could be a special case of #49904 and #49769 but where the hour difference has pushed the date to the next day and then the hour hasn't been printed in the selected date format.

[matriv]

This is also for 6.8 : https://gradle-enterprise.elastic.co/s/srjp5ut4izdoa/console-log?task=:x-pack:plugin:sql:qa:single-node:integTestRunner

[dnhatn]

Anther instance on 6.8: https://gradle-enterprise.elastic.co/s/2ys2zqnvbl5i2

[polyfractal]

Another 6.8: https://gradle-enterprise.elastic.co/s/pd4d4zzbgsm7a

I went ahead and muted in 6.8. Wasn't clear to me if this was failing on other branches, so I left those around

[mark-vieira]

I got one on master of testGettingInvalidDouble failing for a very similar reason. This one also reproduced locally for me. Sounds like this test class simply has trouble with certain timezones? Given this reproduces we should probably just fix this rather than mute anymore tests.

https://gradle-enterprise.elastic.co/s/w7mepl7ylllne/tests/kno2lqcdjxroq-tndm4t2qvqvfm

[costin]

I believe there's a fix for it that just needs to be merged: #51290

[astefan]

@mark-vieira the aforementioned fix is not in yet, because I have doubts about the actual reason for this issue. Both the current issue and #45035 and #48234 have, from SQL point of view, the same cause: same timezone has different offsets in two different Java versions. In a newer Java version (let's say 12), a certain timezone has its offset changed from +1 to +2, for example, compared to Java 8.

And what happens in these tests' failures is that the test itself runs on a Java version while the ES node on which the query is executed against, runs on another Java version.

The aspect I am not sure about is: should the test and ES node have the same Java runtime version? If so (and this was my impression so far), then there is something wrong with our tests infra.
If not, then the workaround in the PR @costin mentioned should fix these failures.

The command to be used when a certain failure is to be reproduced locally specifies the runtime.java version and I assumed this is the same on all the JVMs started in tests. For this particular test it seems there is no runtime specified in the command..., but from logs it's openjdk11 and JAVA_HOME is openjdk13. The actual logs from the tests do not show, though, the java version the ES node has been started with.