Don't count hits via the collector if the hit count can be computed from index stats. #33701
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rom index stats. This is something that we were already doing when sorting by field, which is now also done when sorting by score. As-is this change will speed up top-k `term` queries. This could work for `match_all` queries as well when we implement the `setMinCompetitiveScore` API on their Scorer.
jpountz
added
>enhancement
:Search/Search
Collaborator
|
Pinging @elastic/es-search-aggs |
jimczi
reviewed
Sep 14, 2018
Contributor
jimczi
left a comment
jimczi
left a comment
There was a problem hiding this comment.
I left a comment regarding bwc issue with top docs serialization. LGTM otherwise.
| topDocsSupplier = new CachedSupplier<>(topDocsCollector::topDocs); | ||
| if (hitCount == -1) { | ||
| assert trackTotalHits == false; | ||
| totalHitsSupplier = () -> new TotalHits(topDocsSupplier.get().scoreDocs.length, |
Contributor
There was a problem hiding this comment.
Let's return new TotalHits(0, TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO) for now ? It is translated to -1 by the coordinating node anyway since track_total_hits is false in this case. We also need to handle bwc for mixed search with 6x node. Currently we have an exception if the hit count is greater than 0 and not an exact count so I suspect that this change could break some bwc tests randomly. We can revise this when we add the support for track_total_hits as a number ?
Contributor
Author
There was a problem hiding this comment.
Good point. Will do.
jimczi
approved these changes
Sep 14, 2018
Contributor
Author
|
Thanks @jimczi |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Sep 14, 2018
* master: (24 commits) Only notify ready global checkpoint listeners (elastic#33690) Don't count hits via the collector if the hit count can be computed from index stats. (elastic#33701) Expose retries for CCR fetch failures (elastic#33694) Test fix - Graph vertices could appear in different orders based on map insertion sequence (elastic#33709) Structured audit logging (elastic#31931) Core: Add DateFormatter interface for java time parsing (elastic#33467) [CCR] Check whether the rejected execution exception has the shutdown flag set (elastic#33703) Mute ClusterDisruptionIT#testSendingShardFailure Revert "Mute FullClusterRestartSettingsUpgradeIT" Adjust BWC version on settings upgrade test (elastic#33650) [ML] Allow overrides for some file structure detection decisions (elastic#33630) Adapt skip version for doc_values format deprecation [TEST] wait for no initializing shards [Docs] Minor fix in `has_child` javadoc comment (elastic#33674) Mute FullClusterRestartSettingsUpgradeIT [Kerberos] Add realm name & UPN to user metadata (elastic#33338) [TESTS] Disable specific locales for RestrictedTrustManagerTest (elastic#33299) SQL: Return functions in JDBC driver metadata (elastic#33672) SCRIPTING: Move terms_set Context to its Own Class (elastic#33602) AwaitsFix testRestoreMinmal ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is something that we were already doing when sorting by field, which is
now also done when sorting by score. As-is this change will speed up top-k
termqueries. This could work formatch_allqueries as well when weimplement the
setMinCompetitiveScoreAPI on their Scorer.