[ML] Use a new annotations index for future annotations #79006
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Due to historical bugs a non-negligible proportion of ML users have an annotations index with incorrect mappings. We have published instructions on how to correct the mappings, but the procedure is complicated, and some users prefer to tolerate the lack of annotations functionality rather than attempt the operations necessary to fix the mappings of the existing index. The best way to solve the problem is to create a new annotations index with the correct mappings, and use this for all annotations created from that moment on. This PR implements that solution. The annotations read alias will span the old and new indices in the case where both exist. The annotations write index is adjusted to point only at the new index. Fixes elastic#78439
Author
|
@elasticmachine update branch |
…95/elasticsearch into new_annotations_index_template
Collaborator
|
Pinging @elastic/ml-core (Team:ML) |
droberts195
commented
Oct 14, 2021
|
|
||
| @Before | ||
| public void createComponents() throws Exception { | ||
| waitForMlTemplates(); |
Author
There was a problem hiding this comment.
We don't use a template to create the annotations index, so this should not be needed.
droberts195
commented
Oct 14, 2021
| Settings.builder() | ||
| .put(SETTING_NUMBER_OF_SHARDS, 1) | ||
| .put(SETTING_NUMBER_OF_REPLICAS, 0) | ||
| .put(SETTING_INDEX_HIDDEN, true) |
Author
There was a problem hiding this comment.
This is just making the unit test code reflect production more accurately. It wasn't a bug end users would have seen. Same with the tweaks below.
droberts195
pushed a commit
that referenced
this pull request
Oct 14, 2021
…)" This reverts commit 0af4fe2.
droberts195
pushed a commit
to droberts195/elasticsearch
that referenced
this pull request
Oct 14, 2021
Due to historical bugs a non-negligible proportion of ML users have an annotations index with incorrect mappings. We have published instructions on how to correct the mappings, but the procedure is complicated, and some users prefer to tolerate the lack of annotations functionality rather than attempt the operations necessary to fix the mappings of the existing index. The best way to solve the problem is to create a new annotations index with the correct mappings, and use this for all annotations created from that moment on. This PR implements that solution. The annotations read alias will span the old and new indices in the case where both exist. The annotations write index is adjusted to point only at the new index. Adds back elastic#79006, which was reverted because Kibana was not ready.
droberts195
pushed a commit
that referenced
this pull request
Oct 19, 2021
Due to historical bugs a non-negligible proportion of ML users have an annotations index with incorrect mappings. We have published instructions on how to correct the mappings, but the procedure is complicated, and some users prefer to tolerate the lack of annotations functionality rather than attempt the operations necessary to fix the mappings of the existing index. The best way to solve the problem is to create a new annotations index with the correct mappings, and use this for all annotations created from that moment on. This PR implements that solution. The annotations read alias will span the old and new indices in the case where both exist. The annotations write index is adjusted to point only at the new index. Adds back #79006, which was reverted because Kibana was not ready.
droberts195
pushed a commit
to droberts195/elasticsearch
that referenced
this pull request
Oct 19, 2021
Due to historical bugs a non-negligible proportion of ML users have an annotations index with incorrect mappings. We have published instructions on how to correct the mappings, but the procedure is complicated, and some users prefer to tolerate the lack of annotations functionality rather than attempt the operations necessary to fix the mappings of the existing index. The best way to solve the problem is to create a new annotations index with the correct mappings, and use this for all annotations created from that moment on. This PR implements that solution. The annotations read alias will span the old and new indices in the case where both exist. The annotations write index is adjusted to point only at the new index. Adds back elastic#79006, which was reverted because Kibana was not ready.
droberts195
pushed a commit
that referenced
this pull request
Oct 19, 2021
Due to historical bugs a non-negligible proportion of ML users have an annotations index with incorrect mappings. We have published instructions on how to correct the mappings, but the procedure is complicated, and some users prefer to tolerate the lack of annotations functionality rather than attempt the operations necessary to fix the mappings of the existing index. The best way to solve the problem is to create a new annotations index with the correct mappings, and use this for all annotations created from that moment on. This PR implements that solution. The annotations read alias will span the old and new indices in the case where both exist. The annotations write index is adjusted to point only at the new index. Adds back #79006, which was reverted because Kibana was not ready.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Due to historical bugs a non-negligible proportion of
ML users have an annotations index with incorrect mappings.
We have published instructions on how to correct the
mappings, but the procedure is complicated, and some users
prefer to tolerate the lack of annotations functionality
rather than attempt the operations necessary to fix the
mappings of the existing index.
The best way to solve the problem is to create a new
annotations index with the correct mappings, and use this
for all annotations created from that moment on. This PR
implements that solution. The annotations read alias will
span the old and new indices in the case where both exist.
The annotations write index is adjusted to point only at
the new index.
Fixes #78439