Skip to content

Conversation

@spinscale
Copy link
Contributor

This is the PR of the java-time feature branch, so review can start. The cut-over to java time is a requirement so that we can support nanoseconds properly. Most of the code is replacing the joda time classes with the java time ones - however this cannot be done gradually, thus this one big feature-branch behind this PR.

This mainly converts the field mapper classes plus aggregations and their tests, but also some code in x-pack that relied on joda time.

Note: This cannot be merged yet because we want to push a BWC layer in master/6.x first and then merge this one. I will keep this branch up-to-date until this happens.

Relates #27330

This commit moves the aggregation and mapping code from joda time to
java time.

This includes field mappers, root object mappers, aggregations with date
histograms, query builders and a lot of changes within tests.
Copy link
Contributor

@polyfractal polyfractal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@spinscale
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/2

@spinscale spinscale merged commit daa2ec8 into master Jan 23, 2019
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Jan 23, 2019
…ead-de-duplication

* elastic/master: (24 commits)
  [TEST] Mute MlMappingsUpgradeIT testMappingsUpgrade
  Streamline skip_unavailable handling (elastic#37672)
  Only bootstrap and elect node in current voting configuration (elastic#37712)
  Ensure either success or failure path for SearchOperationListener is called (elastic#37467)
  Target only specific index in update settings test
  Add a note how to benchmark Elasticsearch
  Don't use Groovy's `withDefault` (elastic#37726)
  Adapt SyncedFlushService (elastic#37691)
  Mute FilterAggregatorTests#testRandom
  Switch mapping/aggregations over to java time (elastic#36363)
  [ML] Update ML results mappings on process start (elastic#37706)
  Modify removal_of_types.asciidoc (elastic#37648)
  Fix edge case in PutMappingRequestTests (elastic#37665)
  Use new bulk API endpoint in the docs (elastic#37698)
  Expose sequence number and primary terms in search responses (elastic#37639)
  Remove LicenseServiceClusterNotRecoveredTests (elastic#37528)
  Migrate SpecificMasterNodesIT to Zen2 (elastic#37532)
  Fix MetaStateFormat tests
  Use plain text instead of latexmath
  Fix a typo in a warning message in TestFixturesPlugin (elastic#37631)
  ...
@jsoriano
Copy link
Member

jsoriano commented Jan 25, 2019

Hi @spinscale, there is a test in filebeat failing with ES snapshot and a bisect points to this change.

The failure seen is like this:

{"type":"mapper_parsing_exception","reason":"failed to parse field [suricata.eve.flow.start] of type [date]","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2018-10-03T14:42:44.613469+0000] with format [strict_date_optional_time||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"Text '2018-10-03T14:42:44.613469+0000' could not be parsed, unparsed text found at index 26"}}

Find the ES trace here.

It seems unable to parse 2018-10-03T14:42:44.613469+0000, is it expected?

The test validates this ingest pipeline with these log files.

jsoriano added a commit to jsoriano/beats that referenced this pull request Jan 25, 2019
These dates fail to parse after recent changes in Elasticsearch for date
formatting, see elastic/elasticsearch#36363

Information is still stored in parsed ECS fields.
jsoriano added a commit to elastic/beats that referenced this pull request Jan 25, 2019
These dates fail to parse after recent changes in Elasticsearch for date
formatting, see elastic/elasticsearch#36363

Information is still stored in parsed ECS fields.
@colings86 colings86 deleted the java-time branch May 27, 2020 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Core Core issues without another label >refactoring v7.0.0-beta1

Projects

None yet

Development

Successfully merging this pull request may close these issues.