Avoid changing tsid creating strategy for an index #135514
Conversation
This avoids that the tsid and routing is calculated differently on the same index. Doing so would have the following consequences: * De-duplication would not work, for example, when a client re-transmits a batch as the id (which is based on the tsid) will be different. * When replaying a translog operation, the re-computed tsid and id will differ from the id stored in the translog. This would lead to a failure of the index operation.
elasticsearchmachine
added
external-contributor
| // This can happen if a new dynamic template with time_series_dimension: true is added to an existing index. | ||
| additionalSettings.putList(INDEX_DIMENSIONS.getKey(), List.of()); | ||
| additionalSettings.putList(INDEX_ROUTING_PATH.getKey(), newIndexDimensions); | ||
| throw new IllegalArgumentException( |
There was a problem hiding this comment.
Let's also update the comment above.
There was a problem hiding this comment.
I deleted the comments above and thought that the exception message would serve as the new documentation. What aspects would you like the comment to cover that the exception message doesn't already mention?
There was a problem hiding this comment.
I was referring to the method javadoc, providing an overview of how this works. It'd help readability to include a note there too, imho.
|
I kinda like this. Let's try to be as restrictive as possible. |
|
I like it better, too but didn't make the change earlier due to potential impact on backwards compatibility. Are you worried about that at all? |
|
Not really.. If anything, I'd think it's a best practice to avoid such changes (label => dimension) outside rollovers. More so, we should only have dimensions and metrics at this point, no labels. Maybe worth documenting better. |
This avoids that the tsid and routing is calculated differently on the same index. Doing so would have the following consequences:
This could be seen as a breaking change as operations on backing indices that were allowed before are not allowed after this change. However, this only applies to data streams as
index.dimensionswill never be set on plain time_series indices. For data streams, you first change the template, then optionally the backing indices, and roll over if changing the backing indices doesn't work.