Makes search action cancelable by task management API #20405

imotov · 2016-09-09T18:14:26Z

Long running searches now can be cancelled using standard task cancellation mechanism.

nik9000 · 2016-09-09T18:15:20Z

@imotov, want me to review it?

imotov · 2016-09-09T18:16:33Z

@nik9000 sure - you from Task Management perspective and we will need somebody closer to Lucene ideally.

nik9000 · 2016-09-09T18:17:19Z

core/src/main/java/org/elasticsearch/action/search/SearchTask.java

It'd be cool to be able to list the phase and/or which shards you are waiting for. You could put all kinds of cool stuff in here one day! But for now this seems like the right thing to do.

Yes, that's the plan, but I didn't want to clump everything into a single PR.

nik9000 · 2016-09-12T18:11:14Z

@jpountz, would you like to have a look at this from a Lucene perspective?

jpountz

In general it looks good to me, however I am concerned that wrapping the collector can cause significant slow downs so I'd like to either remove it or make it an opt-in for now and see what other options we have. For instance I'm wondering that we could make the slow down more acceptable by checking whether tasks are cancelled at the bulk scorer level so that we can perform the check less often.

jpountz · 2016-09-20T09:03:20Z

core/src/main/java/org/elasticsearch/transport/TransportService.java

nit: I'd prefer that we move the sendRequest call to the try block and remove the return from the catch block

jpountz · 2016-09-20T09:06:03Z

core/src/main/java/org/elasticsearch/search/internal/SearchContext.java

I don't think task should ever be null at that point? Should we just assert that task is not null?

jpountz · 2016-09-20T09:15:12Z

core/src/main/java/org/elasticsearch/search/query/QueryPhase.java

This part can cause significant slow downs. There may be ways that we can make it better, eg. by doing it at the bulk scorer level so that we can perform the check less often. Can we remove it for now and work on making this part cancellable in another PR. Or at least make it an opt-in for now with a setting?

@jpountz what do you think about this 662e1f5e3a2e9db89511749fe104e922edbb86ad?

I'm not a fan of the name (but I don't have better ideas) but this looks good to me.

jpountz

I left a few more comments about the search part of the PR.

jpountz · 2016-10-13T06:57:36Z

core/src/main/java/org/elasticsearch/search/query/CancellableCollector.java

Can we avoid taking a SearchContext as it makes this class hard to unit test, maybe we could eg. replace the search context with a BooleanSupplier and use a method reference? Then can you add some basic unit tests to this class?

jpountz · 2016-10-13T07:02:54Z

core/src/main/java/org/elasticsearch/search/profile/query/CollectorResult.java

should we be consistent with the number of l? there is only one here while you put 2 in cancellation? (feel free to ignore, it could easily a special case in english that I don't know about!)

jpountz · 2016-10-13T07:03:24Z

core/src/main/java/org/elasticsearch/search/internal/SearchContext.java

Can you make it an actual javadoc and explain the trade-off?

imotov · 2016-10-17T16:47:40Z

@jpountz I have implemented the changes that you have requested and rebased it against the current master since it quite a few things has changed. Would you mind taking another look when you have a chance?

jpountz

I left a question.

jpountz · 2016-10-18T20:30:24Z

core/src/main/java/org/elasticsearch/search/query/CancellableCollector.java

construction contractor? :)

jpountz · 2016-10-18T20:36:11Z

core/src/main/java/org/elasticsearch/search/SearchService.java

why do we only do it in the fetch phase?

sorry I just realized this is the createContext method and not executeFetchPhase as I initially thought.

then I'm wondering if we could have the lowLevelCancellation only on DefaultSearchContext rather than the SearchContext base class?

jpountz · 2016-10-20T09:52:07Z

The search part looks good to me.

nik9000

I left 19 review comments, mostly just comments and a few nits. Please make whatever changes you think are appropriate and merge when ready.

This is a huge start for canceling searches but I think it is worth expanding on the PR's description so folks know exactly what they are getting. Even with it's limitations I think this should be a release highlight for whatever release gets it. 5.1 I think....

nik9000 · 2016-10-20T13:08:02Z

core/src/main/java/org/elasticsearch/transport/TaskAwareTransportRequestHandler.java

I guess you could make this a final class that takes a functional interface as its only parameter and runs it and you could still use lambdas for it. My instincts are that that is marginally better from a design standpoint (composition vs inheritance) and a little easier to read but I'm not sure.

This sounds like an interesting idea, but I think we should do it on the TransportRequestHandler then and should, probably, be another PR.

nik9000 · 2016-10-20T13:10:00Z

core/src/main/java/org/elasticsearch/search/SearchService.java

Can haz javadoc?

nik9000 · 2016-10-20T13:11:45Z

core/src/main/java/org/elasticsearch/search/query/CancellableCollector.java

Maybe mention that this amounts to a volatile read.

It's supplier implementation detail, isn't it? It seems to be wrong place to say something like this. However, the information about how often it will be called can be useful here. I will add that.

Something like "This class assumes that the supplier is fast, with performance on the order of a volatile read." would give a lot of context to the decisions around how to use the Supplier.

nik9000 · 2016-10-20T13:28:31Z

core/src/test/java/org/elasticsearch/search/SearchCancellationIT.java

Can you remove the empty javdoc? We are going to fail the build on those at some point....

nik9000 · 2016-10-20T13:29:03Z

core/src/test/java/org/elasticsearch/search/SearchCancellationIT.java

Can you explain why this is needed in a comment? I'm never sure when to use this and when not to so I figure it'd be nice to have it explained.

Because I want nodes with specific settings and I don't want other tests to run with these settings.

Then could you do something like:

@ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.SUITE) // Changes settings

That'd make it clear why it is needed so folks don't go copying it into their tests without knowing why... 👼

nik9000 · 2016-10-20T13:42:41Z

core/src/test/java/org/elasticsearch/search/SearchCancellationTests.java

If we keep this maybe we should rename it so it doesn't run by default? And javadoc it? It is fairly obvious what it is for but I expect it isn't worth doing it during a regular test run because no one is reading it....

Leftovers, deleted.

nik9000 · 2016-10-20T13:43:05Z

docs/reference/search.asciidoc

s/Long running search/Searchs/ ?

nik9000 · 2016-10-20T13:43:49Z

docs/reference/search.asciidoc

Mention that it only effects searches after the change to the setting.

Are we thinking of automatically opting certain thing into this setting? Maybe we can have search benchmarks that compare?

No, we are thinking about making it generally faster in the next iteration, so we can opt everything in.

nik9000 · 2016-10-20T13:44:38Z

docs/reference/search/profile.asciidoc

I'm sorry about this nightmare.....

I'd love some way to say "just match anything in this position that looks like X" so you don't have to describe the whole path....

nik9000 · 2016-10-20T13:45:45Z

docs/reference/search/profile.asciidoc

Collector?

Like, with the backticks.

Long running searches now can be cancelled using standard task cancellation mechanism.

imotov added >enhancement review :Search/Search Search-related issues that do not fall into other categories :Distributed Coordination/Task Management Issues for anything around the Tasks API - both persistent and node level. v6.0.0-alpha1 v5.1.1 labels Sep 9, 2016

nik9000 reviewed Sep 9, 2016
View reviewed changes

pickypg added the das awesome label Sep 9, 2016

jpountz suggested changes Sep 20, 2016

View reviewed changes

clintongormley mentioned this pull request Sep 23, 2016

Add _top/searches API #12187

Closed

imotov force-pushed the make-searches-cancellable branch from 773b9e1 to 662e1f5 Compare October 13, 2016 04:00

jpountz reviewed Oct 13, 2016

View reviewed changes

imotov force-pushed the make-searches-cancellable branch from 662e1f5 to fcb1be4 Compare October 17, 2016 16:45

jpountz reviewed Oct 18, 2016

View reviewed changes

nik9000 approved these changes Oct 20, 2016

View reviewed changes

jpountz approved these changes Oct 20, 2016

View reviewed changes

nik9000 added the release highlight label Oct 20, 2016

nik9000 removed the :Distributed Coordination/Task Management Issues for anything around the Tasks API - both persistent and node level. label Oct 20, 2016

Makes search action cancelable by task management API

17ad88d

Long running searches now can be cancelled using standard task cancellation mechanism.

imotov force-pushed the make-searches-cancellable branch from 67842a0 to 17ad88d Compare October 26, 2016 01:00

imotov merged commit 17ad88d into elastic:master Oct 26, 2016

eskibars mentioned this pull request Jan 2, 2017

Document detailed parameter for task manager #22407

Closed

eskibars mentioned this pull request Feb 19, 2017

Ability to associate a search task ID #23250

Closed

imotov deleted the make-searches-cancellable branch May 1, 2020 22:25

Makes search action cancelable by task management API #20405

Makes search action cancelable by task management API #20405

Uh oh!

Conversation

imotov commented Sep 9, 2016

Uh oh!

nik9000 commented Sep 9, 2016

Uh oh!

imotov commented Sep 9, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nik9000 commented Sep 12, 2016

Uh oh!

jpountz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpountz Sep 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpountz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

imotov commented Oct 17, 2016

Uh oh!

jpountz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpountz commented Oct 20, 2016

Uh oh!

nik9000 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpountz Sep 20, 2016 •

edited

Loading