Add support for task cancellation to TransportMasterNodeAction #72157
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fcofdez
added
>enhancement
:Distributed Coordination/Task Management
Collaborator
|
Pinging @elastic/es-distributed (Team:Distributed) |
DaveCTurner
reviewed
Apr 23, 2021
Contributor
DaveCTurner
left a comment
DaveCTurner
left a comment
There was a problem hiding this comment.
Looks good, but I think we should also check for cancellation while we're trying to route the request to the master in the first place. Specifically at the top of doStart(), and IMO we should also adjust the predicate passed into the ClusterStateObserver so that it runs doStart() on the first state update after cancellation.
Contributor
Author
|
Thanks for the review!, those are good suggestions. I added the checks to cover these cases too. |
DaveCTurner
reviewed
Apr 23, 2021
| private void executeMasterOperation(Task task, Request request, ClusterState state, | ||
| ActionListener<Response> listener) throws Exception { | ||
| if (task instanceof CancellableTask && ((CancellableTask) task).isCancelled()) { | ||
| throw new CancellationException("Task was cancelled"); |
Contributor
There was a problem hiding this comment.
I don't think we exercise this in the tests any more? At least I removed it and the tests still passed several hundred iterations.
Contributor
Author
There was a problem hiding this comment.
🤦 I've added a new test for this scenario in c1da23f
| final ClusterState.Builder newStateBuilder = ClusterState.builder(stateWithBlock); | ||
|
|
||
| // Either unblock the cluster state or just do an unrelated cluster state change that will check | ||
| // if the task has been cancelled |
Contributor
Author
|
@elasticmachine update branch |
fcofdez
added a commit
to fcofdez/elasticsearch
that referenced
this pull request
Apr 26, 2021
fcofdez
added a commit
that referenced
this pull request
Apr 26, 2021
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
May 11, 2022
In elastic#72157 we made it so that cancelling the task that a `TransportMasterNodeAction` was executing causes the action to fail with a `java.util.concurrent.CancellationException`. This exception is treated as a `500 Internal Server Error` which results in `WARN`-level logs, but cancelling a task is normal and expected behaviour and should not be logged (see elastic#73524). This commit fixes this by using a `TaskCancelledException` instead, since this exception maps to a `400 Bad Request` and generates no logs.
elasticsearchmachine
pushed a commit
that referenced
this pull request
May 11, 2022
In #72157 we made it so that cancelling the task that a `TransportMasterNodeAction` was executing causes the action to fail with a `java.util.concurrent.CancellationException`. This exception is treated as a `500 Internal Server Error` which results in `WARN`-level logs, but cancelling a task is normal and expected behaviour and should not be logged (see #73524). This commit fixes this by using a `TaskCancelledException` instead, since this exception maps to a `400 Bad Request` and generates no logs.
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
May 11, 2022
In elastic#72157 we made it so that cancelling the task that a `TransportMasterNodeAction` was executing causes the action to fail with a `java.util.concurrent.CancellationException`. This exception is treated as a `500 Internal Server Error` which results in `WARN`-level logs, but cancelling a task is normal and expected behaviour and should not be logged (see elastic#73524). This commit fixes this by using a `TaskCancelledException` instead, since this exception maps to a `400 Bad Request` and generates no logs.
elasticsearchmachine
pushed a commit
that referenced
this pull request
May 11, 2022
In #72157 we made it so that cancelling the task that a `TransportMasterNodeAction` was executing causes the action to fail with a `java.util.concurrent.CancellationException`. This exception is treated as a `500 Internal Server Error` which results in `WARN`-level logs, but cancelling a task is normal and expected behaviour and should not be logged (see #73524). This commit fixes this by using a `TaskCancelledException` instead, since this exception maps to a `400 Bad Request` and generates no logs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds support for cancellation on TransportMasterNodeAction.
This is useful to provide cancellation for requests such as
TransportGetMappingsAction