Skip to content

Commit 9aaab6e

Browse files
authored
Save memory on numeric sig terms when not top (#56789)
This saves memory when running numeric significant terms which are not at the top level by merging its collection into numeric terms and relying on the optimization that we made in #55873.
1 parent 96a8ba4 commit 9aaab6e

File tree

15 files changed

+774
-454
lines changed

15 files changed

+774
-454
lines changed

docs/reference/search/profile.asciidoc

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -782,7 +782,7 @@ This yields the following aggregation profile output:
782782
{
783783
"aggregations" : [
784784
{
785-
"type" : "LongTermsAggregator",
785+
"type" : "NumericTermsAggregator",
786786
"description" : "my_scoped_agg",
787787
"time_in_nanos" : 195386,
788788
"breakdown" : {
@@ -796,6 +796,7 @@ This yields the following aggregation profile output:
796796
"collect_count" : 4
797797
},
798798
"debug": {
799+
"result_strategy": "long_terms",
799800
"total_buckets": 4
800801
}
801802
},
@@ -815,7 +816,7 @@ This yields the following aggregation profile output:
815816
},
816817
"children" : [
817818
{
818-
"type" : "LongTermsAggregator",
819+
"type" : "NumericTermsAggregator",
819820
"description" : "my_level_agg",
820821
"time_in_nanos" : 160329,
821822
"breakdown" : {
@@ -829,6 +830,7 @@ This yields the following aggregation profile output:
829830
"collect_count" : 4,
830831
},
831832
"debug": {
833+
"result_strategy": "long_terms",
832834
"total_buckets": 4
833835
}
834836
}
@@ -845,10 +847,10 @@ This yields the following aggregation profile output:
845847
// TESTRESPONSE[s/"id": "\[P6-vulHtQRWuD4YnubWb7A\]\[test\]\[0\]"/"id": $body.profile.shards.0.id/]
846848

847849
From the profile structure we can see that the `my_scoped_agg` is internally
848-
being run as a `LongTermsAggregator` (because the field it is aggregating,
850+
being run as a `NumericTermsAggregator` (because the field it is aggregating,
849851
`likes`, is a numeric field). At the same level, we see a `GlobalAggregator`
850852
which comes from `my_global_agg`. That aggregation then has a child
851-
`LongTermsAggregator` which comes from the second term's aggregation on `likes`.
853+
`NumericTermsAggregator` which comes from the second term's aggregation on `likes`.
852854

853855
The `time_in_nanos` field shows the time executed by each aggregation, and is
854856
inclusive of all children. While the overall time is useful, the `breakdown`

rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/20_terms.yml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -726,8 +726,8 @@ setup:
726726
---
727727
"profiler":
728728
- skip:
729-
version: " - 7.8.99"
730-
reason: debug information added in 7.9.0
729+
version: " - 7.99.99"
730+
reason: debug information added in 7.9.0 (change backporting to 7.9.0)
731731
- do:
732732
bulk:
733733
index: test_1
@@ -782,7 +782,8 @@ setup:
782782
field: number
783783
- match: { aggregations.n_terms.buckets.0.key: 1 }
784784
- match: { aggregations.n_terms.buckets.1.key: 3 }
785-
- match: { profile.shards.0.aggregations.0.type: LongTermsAggregator }
785+
- match: { profile.shards.0.aggregations.0.type: NumericTermsAggregator }
786786
- match: { profile.shards.0.aggregations.0.description: n_terms }
787787
- match: { profile.shards.0.aggregations.0.breakdown.collect_count: 4 }
788+
- match: { profile.shards.0.aggregations.0.debug.result_strategy: long_terms }
788789
- match: { profile.shards.0.aggregations.0.debug.total_buckets: 2 }

server/src/internalClusterTest/java/org/elasticsearch/search/aggregations/bucket/SignificantTermsSignificanceScoreIT.java

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ public void testXContentResponse() throws Exception {
195195

196196
}
197197

198-
public void testDeletesIssue7951() throws Exception {
198+
public void testPopularTermManyDeletedDocs() throws Exception {
199199
String settings = "{\"index.number_of_shards\": 1, \"index.number_of_replicas\": 0}";
200200
assertAcked(prepareCreate(INDEX_NAME).setSettings(settings, XContentType.JSON)
201201
.setMapping("text", "type=keyword", CLASS_FIELD, "type=keyword"));
@@ -237,7 +237,7 @@ public void testDeletesIssue7951() throws Exception {
237237
significantTerms("sig_terms")
238238
.field(TEXT_FIELD)
239239
.minDocCount(1)));
240-
}else
240+
} else
241241
{
242242
request = client().prepareSearch(INDEX_NAME)
243243
.addAggregation(
@@ -477,8 +477,7 @@ public void testScriptScore() throws ExecutionException, InterruptedException, I
477477
.subAggregation(significantText("mySignificantTerms", TEXT_FIELD)
478478
.significanceHeuristic(scriptHeuristic)
479479
.minDocCount(1).shardSize(2).size(2)));
480-
}else
481-
{
480+
} else {
482481
request = client().prepareSearch(INDEX_NAME)
483482
.addAggregation(terms("class").field(CLASS_FIELD)
484483
.subAggregation(significantTerms("mySignificantTerms")

server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/DoubleTerms.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ public class DoubleTerms extends InternalMappedTerms<DoubleTerms, DoubleTerms.Bu
3939
public static final String NAME = "dterms";
4040

4141
static class Bucket extends InternalTerms.Bucket<Bucket> {
42-
private final double term;
42+
double term;
4343

4444
Bucket(double term, long docCount, InternalAggregations aggregations, boolean showDocCountError, long docCountError,
4545
DocValueFormat format) {

server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/DoubleTermsAggregator.java

Lines changed: 0 additions & 78 deletions
This file was deleted.

server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/LongTermsAggregator.java

Lines changed: 0 additions & 196 deletions
This file was deleted.

0 commit comments

Comments
 (0)