Push back on bulk operations #51035
Closed
Push back on bulk operations #51035
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bulk is now nearly fully async, which means it drains its queue faster leading to potentially many active operations waiting for transport responses, fsync or other events. This commit limits the amount of active bulk operations to queue-size + max-threads, reestablishing push back as if bulk were blocking. This also helps not overflood the master with dynamic mapping requests.
henningandersen
added
>bug
WIP
:Distributed Indexing/CRUD
Collaborator
|
Pinging @elastic/es-distributed (:Distributed/CRUD) |
Contributor
Author
|
@elasticmachine run elasticsearch-ci/2 |
rjernst
added
the
Team:Distributed (Obsolete)
Contributor
Author
|
Superseded by #59263. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bulk is now nearly fully async, which means it drains its queue faster
leading to potentially many active operations waiting for transport
responses, fsync or other events.
This commit limits the amount of active bulk operations to queue-size +
max-threads, reestablishing some of the push back.
This also helps not overflood the master with dynamic mapping requests
(though this is not an issue after #51038).