Skip to content

Conversation

@benwtrent
Copy link
Member

This PR adds two validations for the data frame pivot config:

  • That there are no duplicate fields in the group_by or the aggs definitions
  • That there are no fields that are declared as an both an object and not, e.g. both foo.bar.baz and foo.bar

The best case scenario before this PR is that we can automatically determine the mapped type and we prevent the transform from even being started. However, if we rely on the dynamic mapping, index mapping failures will spam the logs until the task eventually fails due to the indexing failures.

Backport of #41895

…41895)

* [ML] verify that there are no duplicate leaf fields in aggs

* addressing pr comments

* addressing PR comments

* optmizing duplication check
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

@benwtrent benwtrent merged commit b23b06d into elastic:7.x May 9, 2019
@benwtrent benwtrent deleted the feature/ml-df-disallow-duplicate-leaf-fields-7.x branch May 9, 2019 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants