Add support for is_write_index flag to data stream aliases. #73462
Conversation
martijnvg
added
>non-issue
v8.0.0
:Data Management/Data streams
elasticmachine
added
the
Team:Data Management
|
Pinging @elastic/es-core-features (Team:Core/Features) |
martijnvg
force-pushed
the
data_stream_aliases_write_flag
branch
from
0e6946b to
da18f13
Compare
danhermann
left a comment
danhermann
left a comment
There was a problem hiding this comment.
LGTM. I left a couple questions below, but none are blocking.
docs/reference/alias.asciidoc
Outdated
There was a problem hiding this comment.
Does this apply also to aliases that point to a single data stream?
There was a problem hiding this comment.
In that case write get forwarded to the write data stream's write index.
There was a problem hiding this comment.
I'm not especially familiar with the snapshot/restore code, but does this work if some of the alias's data streams are present in intersectingDataStreams but the alias's write data stream is not? Also, it looks like it would drop data streams from the alias if they were already present in the cluster but not part of the restore?
There was a problem hiding this comment.
Yes, that is right. I need to fix that (and add a test for it). I wasn't completely finished with this pr yet (meant to open this as a draft pr, but forgot to select it as such).
There was a problem hiding this comment.
Also, it looks like it would drop data streams from the alias if they were already present in the cluster but not part of the restore?
This code just selects the data streams with aliases from the snapshot to restore. As part of the RestoreSnapshotStateTask this is merged with the cluster state the snapshot is restored into (see applyDataStreamRestores() method).
This allows indexing documents into a data stream alias. The ingestion is that forwarded to the write index of the data stream that is marked as write data stream. The `is_write_index` parameter can be used to indicate what the write data stream is, when updating / adding a data steam alias. Relates to elastic#66163
martijnvg
force-pushed
the
data_stream_aliases_write_flag
branch
from
c914bd8 to
a6531cb
Compare
Backport elastic#73462 7.x This allows indexing documents into a data stream alias. The ingestion is that forwarded to the write index of the data stream that is marked as write data stream. The `is_write_index` parameter can be used to indicate what the write data stream is, when updating / adding a data steam alias. Relates to elastic#66163
Backporting #73462 to 7.x branch. This allows indexing documents into a data stream alias. The ingestion is that forwarded to the write index of the data stream that is marked as write data stream. The `is_write_index` parameter can be used to indicate what the write data stream is, when updating / adding a data steam alias. Relates to #66163
4 participants
This allows indexing documents into a data stream alias.
The ingestion is that forwarded to the write index of the data stream
that is marked as write data stream.
The
is_write_indexparameter can be used to indicate what the write data stream is,when updating / adding a data steam alias.
Relates to #66163