Skip to content

Conversation

@danhermann
Copy link
Contributor

This commit adds optional optimistic concurrency control to the API for updating ingest pipelines. This is achieved through the addition of an if_version query parameter similar to the if_seq_no and if_primary_term parameters used for optimistic concurrency control in Elasticsearch. When this parameter is specified, the pipeline update succeeds only if there is an existing pipeline with a version that matches the specified version.

PUT /_ingest/pipeline/my_pipeline?if_version=5
{
  "version": 9,
  "processors": [
    ...
  ]
}

In the example above, the update succeeds only if "my_pipeline" exists with a version of 5. Any version can be specified in the body of the update so long as it is not the existing version. If no version is specified in the body of the update, the version is automatically set to <currentVersion> + 1. Pipelines that do not have a version attribute with a numeric value cannot be updated with the if_version parameter because checking for a null or missing version is not a meaningful guard against concurrent updates.

Note that any pipeline updates that do not include an if_version parameter will be processed unconditionally. Note also that there are no optimizations possible for no-op pipeline updates when the if_version parameter is specified because such an update will always change, at the very least, the version attribute of the pipeline and will therefore never be a no-op update.

Fixes #77031

Backport of #78551

@danhermann danhermann added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP backport v7.16.0 labels Oct 18, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Oct 18, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@danhermann
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/part-1

@danhermann danhermann merged commit 8ebc800 into elastic:7.x Oct 18, 2021
@danhermann danhermann deleted the backport_7x_78551_pipeline_occ branch October 18, 2021 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement Team:Data Management Meta label for data/management team v7.16.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants