Allow closing a write index of a data stream. #70908
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martijnvg
added
the
:Data Management/Data streams
martijnvg
commented
Mar 29, 2021
| assertThat(backingIndices.stream().map(Index::getName).collect(Collectors.toList()), contains(expectedBackingIndexName)); | ||
| } | ||
|
|
||
| public void testSnapshotAndRestoreInPlace() throws Exception { |
Member
Author
There was a problem hiding this comment.
This test will pas when #70934 is merged.
(the restore fails, because after a snapshot was taken, a rollover happened, which created a new backing index, when restoring the version of the data stream where this didn't happen, the restore fails because a unrelated index is in the data stream's namespace)
martijnvg
commented
Mar 29, 2021
| })) { | ||
| @Override | ||
| public ClusterState execute(ClusterState currentState) { | ||
| List<String> writeIndices = new ArrayList<>(); |
Member
Author
There was a problem hiding this comment.
I didn't add a unit test for this, because currently for this code no unit tests exist.
The Cannot freeze write index for data stream yaml test does test this.
elasticmachine
added
the
Team:Data Management
Collaborator
|
Pinging @elastic/es-core-features (Team:Core/Features) |
Contributor
|
@elasticmachine update branch |
danhermann
approved these changes
Mar 29, 2021
Contributor
danhermann
left a comment
danhermann
left a comment
There was a problem hiding this comment.
LGTM. Thanks, @martijnvg!
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Mar 30, 2021
Backport of elastic#70908 to 7.x branch. Prior to this commit when attempting to close a data stream a validation error is returned indicating that it is forbidden to close a write index of a data stream. The idea behind that is to ensure that a data stream always can accept writes. For the same reason deleting a write index is not allowed (the write index can only be deleted when deleting the entire data stream). However closing an index isn't as destructive as deleting an index (an open index request makes the write index available again) and there are other cases where a data stream can't accept writes. For example when primary shards of the write index are not available. So the original reasoning for not allowing to close a write index isn't that strong. On top of this is that this also avoids certain administrative operations from being performed. For example restoring a snapshot containing data streams that already exist in the cluster (in place restore). Closes elastic#70903 elastic#70861
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Mar 30, 2021
Backport of elastic#70908 to 7.12 branch. Prior to this commit when attempting to close a data stream a validation error is returned indicating that it is forbidden to close a write index of a data stream. The idea behind that is to ensure that a data stream always can accept writes. For the same reason deleting a write index is not allowed (the write index can only be deleted when deleting the entire data stream). However closing an index isn't as destructive as deleting an index (an open index request makes the write index available again) and there are other cases where a data stream can't accept writes. For example when primary shards of the write index are not available. So the original reasoning for not allowing to close a write index isn't that strong. On top of this is that this also avoids certain administrative operations from being performed. For example restoring a snapshot containing data streams that already exist in the cluster (in place restore). Closes elastic#70903 elastic#70861
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Mar 30, 2021
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Mar 30, 2021
martijnvg
added a commit
that referenced
this pull request
Mar 30, 2021
This was forgotten as part of #70908
martijnvg
added a commit
that referenced
this pull request
Mar 30, 2021
Backport of #70908 to 7.x branch. Prior to this commit when attempting to close a data stream a validation error is returned indicating that it is forbidden to close a write index of a data stream. The idea behind that is to ensure that a data stream always can accept writes. For the same reason deleting a write index is not allowed (the write index can only be deleted when deleting the entire data stream). However closing an index isn't as destructive as deleting an index (an open index request makes the write index available again) and there are other cases where a data stream can't accept writes. For example when primary shards of the write index are not available. So the original reasoning for not allowing to close a write index isn't that strong. On top of this is that this also avoids certain administrative operations from being performed. For example restoring a snapshot containing data streams that already exist in the cluster (in place restore). Closes #70903 #70861 * Update docs that closing a data stream's write index is allowed See #70908 * iter * iter
martijnvg
added a commit
that referenced
this pull request
Mar 31, 2021
Backport of #70908 to 7.12 branch. Prior to this commit when attempting to close a data stream a validation error is returned indicating that it is forbidden to close a write index of a data stream. The idea behind that is to ensure that a data stream always can accept writes. For the same reason deleting a write index is not allowed (the write index can only be deleted when deleting the entire data stream). However closing an index isn't as destructive as deleting an index (an open index request makes the write index available again) and there are other cases where a data stream can't accept writes. For example when primary shards of the write index are not available. So the original reasoning for not allowing to close a write index isn't that strong. On top of this is that this also avoids certain administrative operations from being performed. For example restoring a snapshot containing data streams that already exist in the cluster (in place restore). Closes #70903 #70861
jrodewig
added a commit
that referenced
this pull request
May 12, 2021
With #70908, you can now close a write index in 7.12.1 and later versions. This removes an outdated reference from the data stream overview docs.
jrodewig
added a commit
that referenced
this pull request
May 12, 2021
jrodewig
added a commit
that referenced
this pull request
May 12, 2021
jrodewig
added a commit
that referenced
this pull request
May 12, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #70903 #70861