Skip to content

Commit f322f53

Browse files
authored
Speed up parsing of large terms queries. (#24210)
The addition of the normalization feature on keywords slowed down the parsing of large `terms` queries since all terms now have to go through normalization. However this can be avoided in the default case that the analyzer is a `keyword` analyzer since all that normalization will do is a UTF8 conversion. Using `Analyzer.normalize` for that is a bit overkill and could be skipped.
1 parent a436597 commit f322f53

File tree

2 files changed

+18
-0
lines changed

2 files changed

+18
-0
lines changed

core/src/main/java/org/elasticsearch/index/mapper/KeywordFieldMapper.java

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,15 @@ public Object valueForDisplay(Object value) {
236236

237237
@Override
238238
protected BytesRef indexedValueForSearch(Object value) {
239+
if (searchAnalyzer() == Lucene.KEYWORD_ANALYZER) {
240+
// keyword analyzer with the default attribute source which encodes terms using UTF8
241+
// in that case we skip normalization, which may be slow if there many terms need to
242+
// parse (eg. large terms query) since Analyzer.normalize involves things like creating
243+
// attributes through reflection
244+
// This if statement will be used whenever a normalizer is NOT configured
245+
return super.indexedValueForSearch(value);
246+
}
247+
239248
if (value == null) {
240249
return null;
241250
}

core/src/test/java/org/elasticsearch/index/mapper/KeywordFieldTypeTests.java

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,4 +150,13 @@ public void testFuzzyQuery() {
150150
() -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true));
151151
assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage());
152152
}
153+
154+
public void testNormalizeQueries() {
155+
MappedFieldType ft = createDefaultFieldType();
156+
ft.setName("field");
157+
ft.setSearchAnalyzer(Lucene.KEYWORD_ANALYZER);
158+
assertEquals(new TermQuery(new Term("field", new BytesRef("FOO"))), ft.termQuery("FOO", null));
159+
ft.setSearchAnalyzer(Lucene.STANDARD_ANALYZER);
160+
assertEquals(new TermQuery(new Term("field", new BytesRef("foo"))), ft.termQuery("FOO", null));
161+
}
153162
}

0 commit comments

Comments
 (0)