diff --git a/docs/reference/mapping/types/geo-shape.asciidoc b/docs/reference/mapping/types/geo-shape.asciidoc index 5ff464da9b995..d2aaa14651279 100644 --- a/docs/reference/mapping/types/geo-shape.asciidoc +++ b/docs/reference/mapping/types/geo-shape.asciidoc @@ -252,6 +252,10 @@ between index size and a reasonable level of precision of 50m at the equator. This allows for indexing tens of millions of shapes without overly bloating the resulting index too much relative to the input size. +[NOTE] +Geo-shape queries on geo-shapes implemented with PrefixTrees will not be executed if +<> is set to false. + [[input-structure]] [float] ==== Input Structure diff --git a/docs/reference/query-dsl.asciidoc b/docs/reference/query-dsl.asciidoc index 58ebe3190a352..51889a5f2c159 100644 --- a/docs/reference/query-dsl.asciidoc +++ b/docs/reference/query-dsl.asciidoc @@ -25,6 +25,27 @@ or to alter their behaviour (such as the Query clauses behave differently depending on whether they are used in <>. + +[[query-dsl-allow-expensive-queries]] +Allow expensive queries:: +Certain types of queries will generally execute slowly due to the way they are implemented, which can affect +the stability of the cluster. Those queries can be categorised as follows: +* Queries that need to do linear scans to identify matches: +** <> +* Queries that have a high up-front cost: +** <> +** <> +** <> without <> +** <> +** <> on <> and <> fields +* <> +* Queries on <> +* Queries that may have a high per-document cost: +** <> +** <> + +The execution of such queries can be prevented by setting the value of the `search.allow_expensive_queries` +setting to `false` (defaults to `true`). -- include::query-dsl/query_filter_context.asciidoc[] @@ -51,4 +72,4 @@ include::query-dsl/minimum-should-match.asciidoc[] include::query-dsl/multi-term-rewrite.asciidoc[] -include::query-dsl/regexp-syntax.asciidoc[] \ No newline at end of file +include::query-dsl/regexp-syntax.asciidoc[] diff --git a/docs/reference/query-dsl/fuzzy-query.asciidoc b/docs/reference/query-dsl/fuzzy-query.asciidoc index bb20e0bd7e720..3a6d4bf786681 100644 --- a/docs/reference/query-dsl/fuzzy-query.asciidoc +++ b/docs/reference/query-dsl/fuzzy-query.asciidoc @@ -97,4 +97,8 @@ adjacent characters (ab → ba). Defaults to `true`. `rewrite`:: (Optional, string) Method used to rewrite the query. For valid values and more -information, see the <>. \ No newline at end of file +information, see the <>. + +==== Notes +Fuzzy queries will not be executed if <> +is set to false. diff --git a/docs/reference/query-dsl/geo-shape-query.asciidoc b/docs/reference/query-dsl/geo-shape-query.asciidoc index 19a22ee103d91..9706b90d82845 100644 --- a/docs/reference/query-dsl/geo-shape-query.asciidoc +++ b/docs/reference/query-dsl/geo-shape-query.asciidoc @@ -161,3 +161,7 @@ and will not match any documents for this query. This can be useful when querying multiple indexes which might have different mappings. When set to `false` (the default value) the query will throw an exception if the field is not mapped. + +==== Notes +Geo-shape queries on geo-shapes implemented with <> will not be executed if +<> is set to false. diff --git a/docs/reference/query-dsl/joining-queries.asciidoc b/docs/reference/query-dsl/joining-queries.asciidoc index 69fcca8690079..140fe111fed79 100644 --- a/docs/reference/query-dsl/joining-queries.asciidoc +++ b/docs/reference/query-dsl/joining-queries.asciidoc @@ -29,4 +29,7 @@ include::has-parent-query.asciidoc[] include::parent-id-query.asciidoc[] - +=== Notes +==== Allow expensive queries +Joining queries will not be executed if <> +is set to false. diff --git a/docs/reference/query-dsl/percolate-query.asciidoc b/docs/reference/query-dsl/percolate-query.asciidoc index 2094be0ccd996..e9d5bcf096270 100644 --- a/docs/reference/query-dsl/percolate-query.asciidoc +++ b/docs/reference/query-dsl/percolate-query.asciidoc @@ -686,3 +686,8 @@ being percolated, as opposed to a single index as we do in examples. There are a allows for fields to be stored in a denser, more efficient way. - Percolate queries do not scale in the same way as other queries, so percolation performance may benefit from using a different index configuration, like the number of primary shards. + +=== Notes +==== Allow expensive queries +Percolate queries will not be executed if <> +is set to false. diff --git a/docs/reference/query-dsl/prefix-query.asciidoc b/docs/reference/query-dsl/prefix-query.asciidoc index 780de433aabc0..8501f0f7d032b 100644 --- a/docs/reference/query-dsl/prefix-query.asciidoc +++ b/docs/reference/query-dsl/prefix-query.asciidoc @@ -64,4 +64,10 @@ GET /_search You can speed up prefix queries using the <> mapping parameter. If enabled, {es} indexes prefixes between 2 and 5 characters in a separate field. This lets {es} run prefix queries more -efficiently at the cost of a larger index. \ No newline at end of file +efficiently at the cost of a larger index. + +[[prefix-query-allow-expensive-queries]] +===== Allow expensive queries +Prefix queries will not be executed if <> +is set to false. However, if <> are enabled, an optimised query is built which +is not considered slow, and will be executed in spite of this setting. diff --git a/docs/reference/query-dsl/query-string-query.asciidoc b/docs/reference/query-dsl/query-string-query.asciidoc index 56eb3b6efb5e5..47523a079341d 100644 --- a/docs/reference/query-dsl/query-string-query.asciidoc +++ b/docs/reference/query-dsl/query-string-query.asciidoc @@ -537,3 +537,9 @@ The example above creates a boolean query: `(blended(terms:[field2:this, field1:this]) blended(terms:[field2:that, field1:that]) blended(terms:[field2:thus, field1:thus]))~2` that matches documents with at least two of the three per-term blended queries. + +==== Notes +===== Allow expensive queries +Query string query can be internally be transformed to a <> which means +that if the prefix queries are disabled as explained <> the query will not be +executed and an exception will be thrown. diff --git a/docs/reference/query-dsl/range-query.asciidoc b/docs/reference/query-dsl/range-query.asciidoc index 73fda308a0b29..ea7e7e9529435 100644 --- a/docs/reference/query-dsl/range-query.asciidoc +++ b/docs/reference/query-dsl/range-query.asciidoc @@ -134,6 +134,11 @@ increases the relevance score. [[range-query-notes]] ==== Notes +[[ranges-on-text-and-keyword]] +===== Using the `range` query with `text` and `keyword` fields +Range queries on <> or <> files will not be executed if +<> is set to false. + [[ranges-on-dates]] ===== Using the `range` query with `date` fields diff --git a/docs/reference/query-dsl/regexp-query.asciidoc b/docs/reference/query-dsl/regexp-query.asciidoc index e92424afbc2a5..522bc68adf8d7 100644 --- a/docs/reference/query-dsl/regexp-query.asciidoc +++ b/docs/reference/query-dsl/regexp-query.asciidoc @@ -86,3 +86,8 @@ regular expressions. `rewrite`:: (Optional, string) Method used to rewrite the query. For valid values and more information, see the <>. + +==== Notes +===== Allow expensive queries +Regexp queries will not be executed if <> +is set to false. diff --git a/docs/reference/query-dsl/script-query.asciidoc b/docs/reference/query-dsl/script-query.asciidoc index 5ed6d4e91f602..cadf3c080df65 100644 --- a/docs/reference/query-dsl/script-query.asciidoc +++ b/docs/reference/query-dsl/script-query.asciidoc @@ -69,3 +69,7 @@ GET /_search } } ---- + +===== Allow expensive queries +Script queries will not be executed if <> +is set to false. diff --git a/docs/reference/query-dsl/script-score-query.asciidoc b/docs/reference/query-dsl/script-score-query.asciidoc index 029cc7469e0f3..d38021119d9c3 100644 --- a/docs/reference/query-dsl/script-score-query.asciidoc +++ b/docs/reference/query-dsl/script-score-query.asciidoc @@ -221,6 +221,10 @@ and default time zone. Also calculations with `now` are not supported. <> are accessible through `script_score` query. +===== Allow expensive queries +Script score queries will not be executed if <> +is set to false. + [[script-score-faster-alt]] ===== Faster alternatives The `script_score` query calculates the score for diff --git a/docs/reference/query-dsl/wildcard-query.asciidoc b/docs/reference/query-dsl/wildcard-query.asciidoc index 5cc1dacfb6efb..3df6570897d6c 100644 --- a/docs/reference/query-dsl/wildcard-query.asciidoc +++ b/docs/reference/query-dsl/wildcard-query.asciidoc @@ -67,4 +67,9 @@ increases the relevance score. `rewrite`:: (Optional, string) Method used to rewrite the query. For valid values and more information, see the -<>. \ No newline at end of file +<>. + +==== Notes +===== Allow expensive queries +Wildcard queries will not be executed if <> +is set to false. diff --git a/modules/mapper-extras/src/test/java/org/elasticsearch/index/mapper/SearchAsYouTypeFieldTypeTests.java b/modules/mapper-extras/src/test/java/org/elasticsearch/index/mapper/SearchAsYouTypeFieldTypeTests.java index 523de91809145..b9cdae13ec139 100644 --- a/modules/mapper-extras/src/test/java/org/elasticsearch/index/mapper/SearchAsYouTypeFieldTypeTests.java +++ b/modules/mapper-extras/src/test/java/org/elasticsearch/index/mapper/SearchAsYouTypeFieldTypeTests.java @@ -26,6 +26,7 @@ import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.index.mapper.SearchAsYouTypeFieldMapper.Defaults; import org.elasticsearch.index.mapper.SearchAsYouTypeFieldMapper.PrefixFieldType; import org.elasticsearch.index.mapper.SearchAsYouTypeFieldMapper.SearchAsYouTypeFieldType; @@ -100,14 +101,19 @@ public void testPrefixQuery() { // this term should be a length that can be rewriteable to a term query on the prefix field final String withinBoundsTerm = "foo"; - assertThat(fieldType.prefixQuery(withinBoundsTerm, CONSTANT_SCORE_REWRITE, null), + assertThat(fieldType.prefixQuery(withinBoundsTerm, CONSTANT_SCORE_REWRITE, randomMockShardContext()), equalTo(new ConstantScoreQuery(new TermQuery(new Term(PREFIX_NAME, withinBoundsTerm))))); // our defaults don't allow a situation where a term can be too small // this term should be too long to be rewriteable to a term query on the prefix field final String longTerm = "toolongforourprefixfieldthistermis"; - assertThat(fieldType.prefixQuery(longTerm, CONSTANT_SCORE_REWRITE, null), + assertThat(fieldType.prefixQuery(longTerm, CONSTANT_SCORE_REWRITE, MOCK_QSC), equalTo(new PrefixQuery(new Term(NAME, longTerm)))); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> fieldType.prefixQuery(longTerm, CONSTANT_SCORE_REWRITE, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[prefix] queries cannot be executed when 'search.allow_expensive_queries' is set to false. " + + "For optimised prefix queries on text fields please enable [index_prefixes].", ee.getMessage()); } } diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasChildQueryBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasChildQueryBuilder.java index 14362afd2e9c6..80f389521e75a 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasChildQueryBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasChildQueryBuilder.java @@ -27,6 +27,7 @@ import org.apache.lucene.search.join.JoinUtil; import org.apache.lucene.search.join.ScoreMode; import org.apache.lucene.search.similarities.Similarity; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.ParsingException; import org.elasticsearch.common.io.stream.StreamInput; @@ -53,6 +54,8 @@ import java.util.Map; import java.util.Objects; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + /** * A query builder for {@code has_child} query. */ @@ -295,6 +298,11 @@ public String getWriteableName() { @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[joining] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + ParentJoinFieldMapper joinFieldMapper = ParentJoinFieldMapper.getMapper(context.getMapperService()); if (joinFieldMapper == null) { if (ignoreUnmapped) { diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasParentQueryBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasParentQueryBuilder.java index 30a2718aab054..7dd3a6dbbd5a4 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasParentQueryBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/query/HasParentQueryBuilder.java @@ -21,6 +21,7 @@ import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.join.ScoreMode; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.ParsingException; import org.elasticsearch.common.io.stream.StreamInput; @@ -45,6 +46,8 @@ import java.util.Map; import java.util.Objects; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + /** * Builder for the 'has_parent' query. */ @@ -158,6 +161,11 @@ public boolean ignoreUnmapped() { @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[joining] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + ParentJoinFieldMapper joinFieldMapper = ParentJoinFieldMapper.getMapper(context.getMapperService()); if (joinFieldMapper == null) { if (ignoreUnmapped) { diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/query/ParentIdQueryBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/query/ParentIdQueryBuilder.java index bc8820c597790..27b98a8c1130a 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/query/ParentIdQueryBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/query/ParentIdQueryBuilder.java @@ -23,6 +23,7 @@ import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.ParsingException; import org.elasticsearch.common.io.stream.StreamInput; @@ -38,6 +39,8 @@ import java.io.IOException; import java.util.Objects; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + public final class ParentIdQueryBuilder extends AbstractQueryBuilder { public static final String NAME = "parent_id"; @@ -153,6 +156,11 @@ public static ParentIdQueryBuilder fromXContent(XContentParser parser) throws IO @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[joining] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + ParentJoinFieldMapper joinFieldMapper = ParentJoinFieldMapper.getMapper(context.getMapperService()); if (joinFieldMapper == null) { if (ignoreUnmapped) { diff --git a/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasChildQueryBuilderTests.java b/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasChildQueryBuilderTests.java index f40b14d5d95ad..d41acf75d1298 100644 --- a/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasChildQueryBuilderTests.java +++ b/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasChildQueryBuilderTests.java @@ -20,7 +20,6 @@ package org.elasticsearch.join.query; import com.carrotsearch.randomizedtesting.generators.RandomPicks; - import org.apache.lucene.index.Term; import org.apache.lucene.search.BooleanClause; import org.apache.lucene.search.BooleanQuery; @@ -32,6 +31,7 @@ import org.apache.lucene.search.join.ScoreMode; import org.apache.lucene.search.similarities.PerFieldSimilarityWrapper; import org.apache.lucene.search.similarities.Similarity; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.Version; import org.elasticsearch.cluster.metadata.IndexMetaData; import org.elasticsearch.common.Strings; @@ -70,6 +70,8 @@ import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.instanceOf; import static org.hamcrest.CoreMatchers.notNullValue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class HasChildQueryBuilderTests extends AbstractQueryTestCase { @@ -361,5 +363,18 @@ public void testExtractInnerHitBuildersWithDuplicate() { queryBuilder.innerHit(new InnerHitBuilder("some_name")); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> InnerHitContextBuilder.extractInnerHits(queryBuilder, Collections.singletonMap("some_name", null))); + assertEquals("[inner_hits] already contains an entry for key [some_name]", e.getMessage()); + } + + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + HasChildQueryBuilder queryBuilder = + hasChildQuery(CHILD_DOC, new TermQueryBuilder("custom_string", "value"), ScoreMode.None); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[joining] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); } } diff --git a/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasParentQueryBuilderTests.java b/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasParentQueryBuilderTests.java index 1af3a056d2fb8..53477e9e8349d 100644 --- a/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasParentQueryBuilderTests.java +++ b/modules/parent-join/src/test/java/org/elasticsearch/join/query/HasParentQueryBuilderTests.java @@ -22,6 +22,7 @@ import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.join.ScoreMode; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.Version; import org.elasticsearch.cluster.metadata.IndexMetaData; import org.elasticsearch.common.Strings; @@ -57,6 +58,8 @@ import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.instanceOf; import static org.hamcrest.CoreMatchers.notNullValue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class HasParentQueryBuilderTests extends AbstractQueryTestCase { private static final String TYPE = "_doc"; @@ -261,5 +264,18 @@ public void testExtractInnerHitBuildersWithDuplicate() { queryBuilder.innerHit(new InnerHitBuilder("some_name")); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> InnerHitContextBuilder.extractInnerHits(queryBuilder, Collections.singletonMap("some_name", null))); + assertEquals("[inner_hits] already contains an entry for key [some_name]", e.getMessage()); + } + + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + HasParentQueryBuilder queryBuilder = new HasParentQueryBuilder( + CHILD_DOC, new WrapperQueryBuilder(new MatchAllQueryBuilder().toString()), false); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[joining] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); } } diff --git a/modules/parent-join/src/test/java/org/elasticsearch/join/query/ParentIdQueryBuilderTests.java b/modules/parent-join/src/test/java/org/elasticsearch/join/query/ParentIdQueryBuilderTests.java index f43214515be0d..27fbce48d9fae 100644 --- a/modules/parent-join/src/test/java/org/elasticsearch/join/query/ParentIdQueryBuilderTests.java +++ b/modules/parent-join/src/test/java/org/elasticsearch/join/query/ParentIdQueryBuilderTests.java @@ -25,6 +25,7 @@ import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TermQuery; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.Version; import org.elasticsearch.cluster.metadata.IndexMetaData; import org.elasticsearch.common.Strings; @@ -48,6 +49,8 @@ import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.instanceOf; import static org.hamcrest.CoreMatchers.notNullValue; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class ParentIdQueryBuilderTests extends AbstractQueryTestCase { @@ -154,4 +157,14 @@ public void testIgnoreUnmapped() throws IOException { assertThat(e.getMessage(), containsString("[" + ParentIdQueryBuilder.NAME + "] no relation found for child [unmapped]")); } + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + ParentIdQueryBuilder queryBuilder = doCreateTestQueryBuilder(); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[joining] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); + } } diff --git a/modules/parent-join/src/test/resources/rest-api-spec/test/11_parent_child.yml b/modules/parent-join/src/test/resources/rest-api-spec/test/11_parent_child.yml index d85f89e768db8..51ca76481cb75 100644 --- a/modules/parent-join/src/test/resources/rest-api-spec/test/11_parent_child.yml +++ b/modules/parent-join/src/test/resources/rest-api-spec/test/11_parent_child.yml @@ -26,6 +26,18 @@ setup: - do: indices.refresh: {} +--- +teardown: + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: null + --- "Parent/child inner hits": - do: @@ -53,3 +65,24 @@ setup: - is_false: hits.hits.0.inner_hits.child.hits.hits.0._nested - gte: { hits.hits.0.inner_hits.child.hits.hits.0._seq_no: 0 } - gte: { hits.hits.0.inner_hits.child.hits.hits.0._primary_term: 1 } + +--- +"HasChild disallow expensive queries": + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + ### Update setting to false + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: "false" + flat_settings: true + + - match: {transient: {search.allow_expensive_queries: "false"}} + + - do: + catch: /\[joining\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + body: { "query": { "has_child": { "type": "child", "query": { "match_all": {} }, "inner_hits": {} } } } diff --git a/modules/parent-join/src/test/resources/rest-api-spec/test/20_parent_join.yml b/modules/parent-join/src/test/resources/rest-api-spec/test/20_parent_join.yml index 0b65c744ec0b2..5798c6f7f2252 100644 --- a/modules/parent-join/src/test/resources/rest-api-spec/test/20_parent_join.yml +++ b/modules/parent-join/src/test/resources/rest-api-spec/test/20_parent_join.yml @@ -51,6 +51,18 @@ setup: - do: indices.refresh: {} +--- +teardown: + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: null + --- "Test basic": - do: @@ -108,3 +120,29 @@ setup: - match: { hits.hits.1._id: "4" } - match: { hits.hits.1._source.join_field.name: "child" } - match: { hits.hits.1._source.join_field.parent: "1" } + +--- +"HasChild disallow expensive queries": + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + ### Update setting to false + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: "false" + flat_settings: true + + - match: {transient: {search.allow_expensive_queries: "false"}} + + - do: + catch: /\[joining\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + body: + sort: [ "id" ] + query: + parent_id: + type: child + id: 1 diff --git a/modules/parent-join/src/test/resources/rest-api-spec/test/30_inner_hits.yml b/modules/parent-join/src/test/resources/rest-api-spec/test/30_inner_hits.yml index 44b1897485f16..53d8a4d8a2a40 100644 --- a/modules/parent-join/src/test/resources/rest-api-spec/test/30_inner_hits.yml +++ b/modules/parent-join/src/test/resources/rest-api-spec/test/30_inner_hits.yml @@ -1,68 +1,118 @@ +--- +setup: + - skip: + version: " - 7.99.99" + reason: "The bug was corrected from 8.0" + + - do: + indices.create: + index: test + body: + mappings: + properties: + entity_type: { "type": "keyword" } + join_field: { "type": "join", "relations": { "question": "answer", "person": "address" } } + settings: + number_of_shards: 1 + + - do: + index: + index: test + id: 1 + body: { "join_field": { "name": "question" }, "entity_type": "question" } + + - do: + index: + index: test + id: 2 + routing: 1 + body: { "join_field": { "name": "answer", "parent": 1} , "entity_type": "answer" } + + - do: + index: + index: test + id: 3 + body: { "join_field": { "name": "person" }, "entity_type": "person" } + + - do: + index: + index: test + routing: 3 + id: 4 + body: { "join_field": { "name": "address", "parent": 3 }, "entity_type": "address" } + + - do: + indices.refresh: {} + +--- +teardown: + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: null + --- "Test two sub-queries with only one having inner_hits": - - skip: - version: " - 7.99.99" - reason: "The bug was corrected from 8.0" - - - do: - indices.create: - index: test - body: - mappings: - properties: - entity_type: { "type": "keyword" } - join_field: { "type": "join", "relations": { "question": "answer", "person" : "address" } } - settings: - number_of_shards: 1 - - - do: - index: - index: test - id: 1 - body: { "join_field": { "name": "question" }, "entity_type": "question" } - - - do: - index: - index: test - id: 2 - routing: 1 - body: { "join_field": { "name": "answer", "parent": 1} , "entity_type": "answer" } - - - do: - index: - index: test - id: 3 - body: { "join_field": { "name": "person" }, "entity_type": "person" } - - - do: - index: - index: test - routing: 3 - id: 4 - body: { "join_field": { "name": "address", "parent": 3 }, "entity_type": "address" } - - - do: - indices.refresh: {} - - - do: - search: - index: test - body: - query: - bool: - should: - - term: - entity_type: person - - has_parent: - parent_type: question - query: - match_all: {} - inner_hits: {} - - - - match: { hits.total.value: 2 } - - match: { hits.hits.0._id: "3" } - - match: { hits.hits.0.inner_hits.question.hits.total.value: 0} - - match: { hits.hits.1._id: "2" } - - match: { hits.hits.1.inner_hits.question.hits.total.value: 1} - - match: { hits.hits.1.inner_hits.question.hits.hits.0._id: "1"} + - skip: + version: " - 7.99.99" + reason: "The bug was corrected from 8.0" + + - do: + search: + index: test + body: + query: + bool: + should: + - term: + entity_type: person + - has_parent: + parent_type: question + query: + match_all: {} + inner_hits: {} + + + - match: { hits.total.value: 2 } + - match: { hits.hits.0._id: "3" } + - match: { hits.hits.0.inner_hits.question.hits.total.value: 0} + - match: { hits.hits.1._id: "2" } + - match: { hits.hits.1.inner_hits.question.hits.total.value: 1} + - match: { hits.hits.1.inner_hits.question.hits.hits.0._id: "1"} + +--- +"HasParent disallow expensive queries": + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + ### Update setting to false + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: "false" + flat_settings: true + + - match: {transient: {search.allow_expensive_queries: "false"}} + + - do: + catch: /\[joining\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + bool: + should: + - term: + entity_type: person + - has_parent: + parent_type: question + query: + match_all: {} + inner_hits: {} diff --git a/modules/percolator/src/main/java/org/elasticsearch/percolator/PercolateQueryBuilder.java b/modules/percolator/src/main/java/org/elasticsearch/percolator/PercolateQueryBuilder.java index 1f083aa207613..57a49dbed4f65 100644 --- a/modules/percolator/src/main/java/org/elasticsearch/percolator/PercolateQueryBuilder.java +++ b/modules/percolator/src/main/java/org/elasticsearch/percolator/PercolateQueryBuilder.java @@ -87,6 +87,8 @@ import java.util.Objects; import java.util.function.Supplier; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + public class PercolateQueryBuilder extends AbstractQueryBuilder { public static final String NAME = "percolate"; @@ -491,6 +493,11 @@ protected QueryBuilder doRewrite(QueryRewriteContext queryShardContext) { @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[percolate] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + // Call nowInMillis() so that this query becomes un-cacheable since we // can't be sure that it doesn't use now or scripts context.nowInMillis(); diff --git a/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolateQueryBuilderTests.java b/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolateQueryBuilderTests.java index 0c914295f9fc8..f5decb7303e64 100644 --- a/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolateQueryBuilderTests.java +++ b/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolateQueryBuilderTests.java @@ -20,6 +20,7 @@ package org.elasticsearch.percolator; import org.apache.lucene.search.Query; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.ResourceNotFoundException; import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest; import org.elasticsearch.action.get.GetRequest; @@ -57,6 +58,8 @@ import static org.elasticsearch.index.seqno.SequenceNumbers.UNASSIGNED_SEQ_NO; import static org.hamcrest.Matchers.equalTo; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class PercolateQueryBuilderTests extends AbstractQueryTestCase { @@ -341,4 +344,14 @@ public void testSettingNameWhileRewritingWhenDocumentSupplierAndSourceNotNull() assertNotEquals(rewrittenQueryBuilder, percolateQueryBuilder); } + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + PercolateQueryBuilder queryBuilder = doCreateTestQueryBuilder(true); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[percolate] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); + } } diff --git a/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolatorQuerySearchIT.java b/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolatorQuerySearchIT.java index c726d203087ed..75a9d5d105d30 100644 --- a/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolatorQuerySearchIT.java +++ b/modules/percolator/src/test/java/org/elasticsearch/percolator/PercolatorQuerySearchIT.java @@ -19,11 +19,14 @@ package org.elasticsearch.percolator; import org.apache.lucene.search.join.ScoreMode; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest; import org.elasticsearch.action.search.MultiSearchResponse; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.common.bytes.BytesArray; import org.elasticsearch.common.bytes.BytesReference; import org.elasticsearch.common.geo.GeoPoint; +import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.unit.DistanceUnit; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentFactory; @@ -37,6 +40,7 @@ import org.elasticsearch.search.sort.SortOrder; import org.elasticsearch.test.ESIntegTestCase; +import java.io.IOException; import java.util.Arrays; import java.util.Collections; @@ -875,4 +879,55 @@ public void testPercolatorQueryViaMultiSearch() throws Exception { assertThat(item.getFailureMessage(), containsString("[test/6] couldn't be found")); } + public void testDallowExpensiveQueries() throws IOException { + try { + assertAcked(client().admin().indices().prepareCreate("test") + .setMapping("id", "type=keyword", "field1", "type=keyword", "query", "type=percolator") + ); + + client().prepareIndex("test").setId("1") + .setSource(jsonBuilder().startObject() + .field("id", "1") + .field("query", matchQuery("field1", "value")).endObject()) + .get(); + refresh(); + + // Execute with search.allow_expensive_queries = null => default value = false => success + BytesReference source = BytesReference.bytes(jsonBuilder().startObject().field("field1", "value").endObject()); + SearchResponse response = client().prepareSearch() + .setQuery(new PercolateQueryBuilder("query", source, XContentType.JSON)) + .get(); + assertHitCount(response, 1); + assertThat(response.getHits().getAt(0).getId(), equalTo("1")); + assertThat(response.getHits().getAt(0).getFields().get("_percolator_document_slot").getValue(), equalTo(0)); + + // Set search.allow_expensive_queries to "false" => assert failure + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", false)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> client().prepareSearch() + .setQuery(new PercolateQueryBuilder("query", source, XContentType.JSON)) + .get()); + assertEquals("[percolate] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getCause().getMessage()); + + // Set search.allow_expensive_queries setting to "true" ==> success + updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + + response = client().prepareSearch() + .setQuery(new PercolateQueryBuilder("query", source, XContentType.JSON)) + .get(); + assertHitCount(response, 1); + assertThat(response.getHits().getAt(0).getId(), equalTo("1")); + assertThat(response.getHits().getAt(0).getFields().get("_percolator_document_slot").getValue(), equalTo(0)); + } finally { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + } + } } diff --git a/plugins/analysis-icu/src/main/java/org/elasticsearch/index/mapper/ICUCollationKeywordFieldMapper.java b/plugins/analysis-icu/src/main/java/org/elasticsearch/index/mapper/ICUCollationKeywordFieldMapper.java index 4b29d314356df..883468941a5d8 100644 --- a/plugins/analysis-icu/src/main/java/org/elasticsearch/index/mapper/ICUCollationKeywordFieldMapper.java +++ b/plugins/analysis-icu/src/main/java/org/elasticsearch/index/mapper/ICUCollationKeywordFieldMapper.java @@ -158,7 +158,7 @@ protected BytesRef indexedValueForSearch(Object value) { @Override public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int maxExpansions, - boolean transpositions) { + boolean transpositions, QueryShardContext context) { throw new UnsupportedOperationException("[fuzzy] queries are not supported on [" + CONTENT_TYPE + "] fields."); } diff --git a/plugins/analysis-icu/src/test/java/org/elasticsearch/index/mapper/CollationFieldTypeTests.java b/plugins/analysis-icu/src/test/java/org/elasticsearch/index/mapper/CollationFieldTypeTests.java index a261e8b3b7e9a..c44ead42557ae 100644 --- a/plugins/analysis-icu/src/test/java/org/elasticsearch/index/mapper/CollationFieldTypeTests.java +++ b/plugins/analysis-icu/src/test/java/org/elasticsearch/index/mapper/CollationFieldTypeTests.java @@ -28,6 +28,7 @@ import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.unit.Fuzziness; import org.elasticsearch.index.mapper.ICUCollationKeywordFieldMapper.CollationFieldType; import org.elasticsearch.index.mapper.MappedFieldType.Relation; @@ -101,32 +102,36 @@ public void testRegexpQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); - expectThrows(UnsupportedOperationException.class, - () -> ft.regexpQuery("foo.*", 0, 10, null, null)); + UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, + () -> ft.regexpQuery("foo.*", 0, 10, null, randomMockShardContext())); + assertEquals("[regexp] queries are not supported on [icu_collation_keyword] fields.", e.getMessage()); } public void testFuzzyQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); - expectThrows(UnsupportedOperationException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true)); + UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, + () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, randomMockShardContext())); + assertEquals("[fuzzy] queries are not supported on [icu_collation_keyword] fields.", e.getMessage()); } public void testPrefixQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); - expectThrows(UnsupportedOperationException.class, - () -> ft.prefixQuery("prefix", null, null)); + UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, + () -> ft.prefixQuery("prefix", null, randomMockShardContext())); + assertEquals("[prefix] queries are not supported on [icu_collation_keyword] fields.", e.getMessage()); } public void testWildcardQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); - expectThrows(UnsupportedOperationException.class, - () -> ft.wildcardQuery("foo*", null, null)); + UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, + () -> ft.wildcardQuery("foo*", null, randomMockShardContext())); + assertEquals("[wildcard] queries are not supported on [icu_collation_keyword] fields.", e.getMessage()); } public void testRangeQuery() { @@ -143,11 +148,16 @@ public void testRangeQuery() { TermRangeQuery expected = new TermRangeQuery("field", new BytesRef(aKey.bytes, 0, aKey.size), new BytesRef(bKey.bytes, 0, bKey.size), false, false); - assertEquals(expected, ft.rangeQuery("a", "b", false, false, null, null, null, null)); + assertEquals(expected, ft.rangeQuery("a", "b", false, false, null, null, null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.rangeQuery("a", "b", true, true, null, null, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[range] queries on [text] or [keyword] fields cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", ee.getMessage()); ft.setIndexOptions(IndexOptions.NONE); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, - () -> ft.rangeQuery("a", "b", false, false, null, null, null, null)); + () -> ft.rangeQuery("a", "b", false, false, null, null, null, MOCK_QSC)); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); } } diff --git a/rest-api-spec/src/main/resources/rest-api-spec/test/search/320_disallow_queries.yml b/rest-api-spec/src/main/resources/rest-api-spec/test/search/320_disallow_queries.yml new file mode 100644 index 0000000000000..ceb8bdeb1115c --- /dev/null +++ b/rest-api-spec/src/main/resources/rest-api-spec/test/search/320_disallow_queries.yml @@ -0,0 +1,330 @@ +--- +setup: + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + - do: + indices.create: + index: test + body: + mappings: + properties: + text: + type: text + analyzer: standard + fields: + raw: + type: keyword + nested1: + type: nested + + - do: + bulk: + refresh: true + body: + - '{"index": {"_index": "test", "_id": "1"}}' + - '{"text" : "Some like it hot, some like it cold", "nested1": [{"foo": "bar1"}]}' + - '{"index": {"_index": "test", "_id": "2"}}' + - '{"text" : "Its cold outside, theres no kind of atmosphere", "nested1": [{"foo": "bar2"}]}' + - '{"index": {"_index": "test", "_id": "3"}}' + - '{"text" : "Baby its cold there outside", "nested1": [{"foo": "bar3"}]}' + - '{"index": {"_index": "test", "_id": "4"}}' + - '{"text" : "Outside it is cold and wet", "nested1": [{"foo": "bar4"}]}' + +--- +teardown: + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: null + +--- +"Test disallow expensive queries": + - skip: + version: " - 7.99.99" + reason: "implemented in 8.0.0" + + ### Check for initial setting = null -> false + - do: + cluster.get_settings: + flat_settings: true + + - match: {search.allow_expensive_queries: null} + + ### Prefix + - do: + search: + index: test + body: + query: + prefix: + text: + value: out + + - match: { hits.total.value: 3 } + + ### Fuzzy + - do: + search: + index: test + body: + query: + fuzzy: + text: + value: outwide + + - match: { hits.total.value: 3 } + + + ### Regexp + - do: + search: + index: test + body: + query: + regexp: + text: + value: .*ou.*id.* + + - match: { hits.total.value: 3 } + + ### Wildcard + - do: + search: + index: test + body: + query: + wildcard: + text: + value: out?ide + + - match: { hits.total.value: 3 } + + ### Range on text + - do: + search: + index: test + body: + query: + range: + text: + gte: "theres" + + - match: { hits.total.value: 2 } + + ### Range on keyword + - do: + search: + index: test + body: + query: + range: + text.raw: + gte : "Outside it is cold and wet" + + - match: { hits.total.value: 2 } + + ### Nested + - do: + search: + index: test + body: + query: + nested: + path: "nested1" + query: + bool: + must: [{"match": {"nested1.foo": "bar2"}}] + + - match: { hits.total.value: 1 } + + ### Update setting to false + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: "false" + flat_settings: true + + - match: {transient: {search.allow_expensive_queries: "false"}} + + ### Prefix + - do: + catch: /\[prefix\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false. For optimised prefix queries on text fields please enable \[index_prefixes\]./ + search: + index: test + body: + query: + prefix: + text: + value: out + + ### Fuzzy + - do: + catch: /\[fuzzy\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + fuzzy: + text: + value: outwide + + ### Regexp + - do: + catch: /\[regexp\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + regexp: + text: + value: .*ou.*id.* + + ### Wildcard + - do: + catch: /\[wildcard\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + wildcard: + text: + value: out?ide + + ### Range on text + - do: + catch: /\[range\] queries on \[text\] or \[keyword\] fields cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + range: + text: + gte: "theres" + + ### Range on keyword + - do: + catch: /\[range\] queries on \[text\] or \[keyword\] fields cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + range: + text.raw: + gte : "Outside it is cold and wet" + + ### Nested + - do: + catch: /\[joining\] queries cannot be executed when \'search.allow_expensive_queries\' is set to false./ + search: + index: test + body: + query: + nested: + path: "nested1" + query: + bool: + must: [{"match" : {"nested1.foo" : "bar2"}}] + + ### Revert setting to true + - do: + cluster.put_settings: + body: + transient: + search.allow_expensive_queries: "true" + flat_settings: true + + - match: {transient: {search.allow_expensive_queries: "true"}} + + ### Prefix + - do: + search: + index: test + body: + query: + prefix: + text: + value: out + + - match: { hits.total.value: 3 } + + ### Fuzzy + - do: + search: + index: test + body: + query: + fuzzy: + text: + value: outwide + + - match: { hits.total.value: 3 } + + ### Regexp + - do: + search: + index: test + body: + query: + regexp: + text: + value: .*ou.*id.* + + - match: { hits.total.value: 3 } + + ### Wildcard + - do: + search: + index: test + body: + query: + wildcard: + text: + value: out?ide + + - match: { hits.total.value: 3 } + + ### Range on text + - do: + search: + index: test + body: + query: + range: + text: + gte: "theres" + + - match: { hits.total.value: 2 } + + ### Range on keyword + - do: + search: + index: test + body: + query: + range: + text.raw: + gte: "Outside it is cold and wet" + + - match: { hits.total.value: 2 } + + ### Nested + - do: + search: + index: test + body: + query: + nested: + path: "nested1" + query: + bool: + must: [{"match": {"nested1.foo": "bar2"}}] + + - match: { hits.total.value: 1 } diff --git a/server/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java b/server/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java index 960fbbf5571d7..4861cea792144 100644 --- a/server/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java +++ b/server/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java @@ -390,6 +390,7 @@ public void apply(Settings value, Settings current, Settings previous) { SearchService.DEFAULT_KEEPALIVE_SETTING, SearchService.KEEPALIVE_INTERVAL_SETTING, SearchService.MAX_KEEPALIVE_SETTING, + SearchService.ALLOW_EXPENSIVE_QUERIES, MultiBucketConsumerService.MAX_BUCKET_SETTING, SearchService.LOW_LEVEL_CANCELLATION_SETTING, SearchService.MAX_OPEN_SCROLL_CONTEXT, diff --git a/server/src/main/java/org/elasticsearch/index/IndexModule.java b/server/src/main/java/org/elasticsearch/index/IndexModule.java index c70e4c8f44154..3660d2097c1d8 100644 --- a/server/src/main/java/org/elasticsearch/index/IndexModule.java +++ b/server/src/main/java/org/elasticsearch/index/IndexModule.java @@ -130,6 +130,7 @@ public final class IndexModule { private final List searchOperationListeners = new ArrayList<>(); private final List indexOperationListeners = new ArrayList<>(); private final AtomicBoolean frozen = new AtomicBoolean(false); + private final BooleanSupplier allowExpensiveQueries; /** * Construct the index module for the index with the specified index settings. The index module contains extension points for plugins @@ -144,13 +145,15 @@ public IndexModule( final IndexSettings indexSettings, final AnalysisRegistry analysisRegistry, final EngineFactory engineFactory, - final Map directoryFactories) { + final Map directoryFactories, + final BooleanSupplier allowExpensiveQueries) { this.indexSettings = indexSettings; this.analysisRegistry = analysisRegistry; this.engineFactory = Objects.requireNonNull(engineFactory); this.searchOperationListeners.add(new SearchSlowLog(indexSettings)); this.indexOperationListeners.add(new IndexingSlowLog(indexSettings)); this.directoryFactories = Collections.unmodifiableMap(directoryFactories); + this.allowExpensiveQueries = allowExpensiveQueries; } /** @@ -424,7 +427,7 @@ public IndexService newIndexService( new SimilarityService(indexSettings, scriptService, similarities), shardStoreDeleter, indexAnalyzers, engineFactory, circuitBreakerService, bigArrays, threadPool, scriptService, clusterService, client, queryCache, directoryFactory, eventListener, readerWrapperFactory, mapperRegistry, indicesFieldDataCache, searchOperationListeners, - indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled); + indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled, allowExpensiveQueries); success = true; return indexService; } finally { diff --git a/server/src/main/java/org/elasticsearch/index/IndexService.java b/server/src/main/java/org/elasticsearch/index/IndexService.java index 6512a7f1a6b3f..c52779d645970 100644 --- a/server/src/main/java/org/elasticsearch/index/IndexService.java +++ b/server/src/main/java/org/elasticsearch/index/IndexService.java @@ -59,8 +59,8 @@ import org.elasticsearch.index.fielddata.IndexFieldDataCache; import org.elasticsearch.index.fielddata.IndexFieldDataService; import org.elasticsearch.index.mapper.MapperService; -import org.elasticsearch.index.query.SearchIndexNameMatcher; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.index.query.SearchIndexNameMatcher; import org.elasticsearch.index.seqno.RetentionLeaseSyncer; import org.elasticsearch.index.shard.IndexEventListener; import org.elasticsearch.index.shard.IndexShard; @@ -126,6 +126,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust private final IndexSettings indexSettings; private final List searchOperationListeners; private final List indexingOperationListeners; + private final BooleanSupplier allowExpensiveQueries; private volatile AsyncRefreshTask refreshTask; private volatile AsyncTranslogFSync fsyncTask; private volatile AsyncGlobalCheckpointTask globalCheckpointTask; @@ -166,8 +167,10 @@ public IndexService( List searchOperationListeners, List indexingOperationListeners, NamedWriteableRegistry namedWriteableRegistry, - BooleanSupplier idFieldDataEnabled) { + BooleanSupplier idFieldDataEnabled, + BooleanSupplier allowExpensiveQueries) { super(indexSettings); + this.allowExpensiveQueries = allowExpensiveQueries; this.indexSettings = indexSettings; this.xContentRegistry = xContentRegistry; this.similarityService = similarityService; @@ -568,7 +571,7 @@ public QueryShardContext newQueryShardContext(int shardId, IndexSearcher searche return new QueryShardContext( shardId, indexSettings, bigArrays, indexCache.bitsetFilterCache(), indexFieldData::getForField, mapperService(), similarityService(), scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, clusterAlias, - indexNameMatcher); + indexNameMatcher, allowExpensiveQueries); } /** diff --git a/server/src/main/java/org/elasticsearch/index/mapper/MappedFieldType.java b/server/src/main/java/org/elasticsearch/index/mapper/MappedFieldType.java index 6cc5636e5ddff..e6e31e3379fb2 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/MappedFieldType.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/MappedFieldType.java @@ -26,6 +26,7 @@ import org.apache.lucene.index.PrefixCodedTerms; import org.apache.lucene.index.PrefixCodedTerms.TermIterator; import org.apache.lucene.index.Term; +import org.apache.lucene.queries.intervals.IntervalsSource; import org.apache.lucene.search.BooleanClause.Occur; import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.BoostQuery; @@ -34,7 +35,6 @@ import org.apache.lucene.search.Query; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; -import org.apache.lucene.queries.intervals.IntervalsSource; import org.apache.lucene.search.spans.SpanMultiTermQueryWrapper; import org.apache.lucene.search.spans.SpanQuery; import org.apache.lucene.util.BytesRef; @@ -350,7 +350,8 @@ public Query rangeQuery( throw new IllegalArgumentException("Field [" + name + "] of type [" + typeName() + "] does not support range queries"); } - public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int maxExpansions, boolean transpositions) { + public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int maxExpansions, boolean transpositions, + QueryShardContext context) { throw new IllegalArgumentException("Can only use fuzzy queries on keyword and text fields - not on [" + name + "] which is of type [" + typeName() + "]"); } diff --git a/server/src/main/java/org/elasticsearch/index/mapper/StringFieldType.java b/server/src/main/java/org/elasticsearch/index/mapper/StringFieldType.java index cde8e392dabb8..4ddda3df0af2d 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/StringFieldType.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/StringFieldType.java @@ -31,6 +31,7 @@ import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.search.WildcardQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.lucene.BytesRefs; import org.elasticsearch.common.unit.Fuzziness; import org.elasticsearch.index.query.QueryShardContext; @@ -38,6 +39,8 @@ import java.util.List; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + /** Base class for {@link MappedFieldType} implementations that use the same * representation for internal index terms as the external representation so * that partial matching queries such as prefix, wildcard and fuzzy queries @@ -62,7 +65,11 @@ public Query termsQuery(List values, QueryShardContext context) { @Override public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int maxExpansions, - boolean transpositions) { + boolean transpositions, QueryShardContext context) { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[fuzzy] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } failIfNotIndexed(); return new FuzzyQuery(new Term(name(), indexedValueForSearch(value)), fuzziness.asDistance(BytesRefs.toString(value)), prefixLength, maxExpansions, transpositions); @@ -70,6 +77,11 @@ public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int @Override public Query prefixQuery(String value, MultiTermQuery.RewriteMethod method, QueryShardContext context) { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[prefix] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false. For optimised prefix queries on text " + + "fields please enable [index_prefixes]."); + } failIfNotIndexed(); PrefixQuery query = new PrefixQuery(new Term(name(), indexedValueForSearch(value))); if (method != null) { @@ -84,6 +96,11 @@ public Query wildcardQuery(String value, MultiTermQuery.RewriteMethod method, Qu if (termQuery instanceof MatchNoDocsQuery || termQuery instanceof MatchAllDocsQuery) { return termQuery; } + + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[wildcard] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } Term term = MappedFieldType.extractTerm(termQuery); WildcardQuery query = new WildcardQuery(term); @@ -94,6 +111,10 @@ public Query wildcardQuery(String value, MultiTermQuery.RewriteMethod method, Qu @Override public Query regexpQuery(String value, int flags, int maxDeterminizedStates, MultiTermQuery.RewriteMethod method, QueryShardContext context) { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[regexp] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } failIfNotIndexed(); RegexpQuery query = new RegexpQuery(new Term(name(), indexedValueForSearch(value)), flags, maxDeterminizedStates); if (method != null) { @@ -104,6 +125,10 @@ public Query regexpQuery(String value, int flags, int maxDeterminizedStates, @Override public Query rangeQuery(Object lowerTerm, Object upperTerm, boolean includeLower, boolean includeUpper, QueryShardContext context) { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[range] queries on [text] or [keyword] fields cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } failIfNotIndexed(); return new TermRangeQuery(name(), lowerTerm == null ? null : indexedValueForSearch(lowerTerm), diff --git a/server/src/main/java/org/elasticsearch/index/query/FuzzyQueryBuilder.java b/server/src/main/java/org/elasticsearch/index/query/FuzzyQueryBuilder.java index 954107c656086..8df0fec044124 100644 --- a/server/src/main/java/org/elasticsearch/index/query/FuzzyQueryBuilder.java +++ b/server/src/main/java/org/elasticsearch/index/query/FuzzyQueryBuilder.java @@ -328,7 +328,7 @@ protected Query doToQuery(QueryShardContext context) throws IOException { String rewrite = this.rewrite; MappedFieldType fieldType = context.fieldMapper(fieldName); if (fieldType != null) { - query = fieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions); + query = fieldType.fuzzyQuery(value, fuzziness, prefixLength, maxExpansions, transpositions, context); } if (query == null) { int maxEdits = fuzziness.asDistance(BytesRefs.toString(value)); diff --git a/server/src/main/java/org/elasticsearch/index/query/LegacyGeoShapeQueryProcessor.java b/server/src/main/java/org/elasticsearch/index/query/LegacyGeoShapeQueryProcessor.java index 14307d846b67e..20bbb65f34f39 100644 --- a/server/src/main/java/org/elasticsearch/index/query/LegacyGeoShapeQueryProcessor.java +++ b/server/src/main/java/org/elasticsearch/index/query/LegacyGeoShapeQueryProcessor.java @@ -26,6 +26,7 @@ import org.apache.lucene.spatial.prefix.RecursivePrefixTreeStrategy; import org.apache.lucene.spatial.query.SpatialArgs; import org.apache.lucene.spatial.query.SpatialOperation; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.geo.ShapeRelation; import org.elasticsearch.common.geo.SpatialStrategy; import org.elasticsearch.common.geo.builders.CircleBuilder; @@ -59,6 +60,8 @@ import java.util.ArrayList; import java.util.List; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + public class LegacyGeoShapeQueryProcessor implements AbstractGeometryFieldMapper.QueryProcessor { private AbstractGeometryFieldMapper.AbstractGeometryFieldType ft; @@ -74,6 +77,11 @@ public Query process(Geometry shape, String fieldName, ShapeRelation relation, Q @Override public Query process(Geometry shape, String fieldName, SpatialStrategy strategy, ShapeRelation relation, QueryShardContext context) { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[geo-shape] queries on [PrefixTree geo shapes] cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + LegacyGeoShapeFieldMapper.GeoShapeFieldType shapeFieldType = (LegacyGeoShapeFieldMapper.GeoShapeFieldType) ft; SpatialStrategy spatialStrategy = shapeFieldType.strategy(); if (strategy != null) { diff --git a/server/src/main/java/org/elasticsearch/index/query/NestedQueryBuilder.java b/server/src/main/java/org/elasticsearch/index/query/NestedQueryBuilder.java index fecf5c8407e98..a560af6826af9 100644 --- a/server/src/main/java/org/elasticsearch/index/query/NestedQueryBuilder.java +++ b/server/src/main/java/org/elasticsearch/index/query/NestedQueryBuilder.java @@ -34,6 +34,7 @@ import org.apache.lucene.search.join.BitSetProducer; import org.apache.lucene.search.join.ParentChildrenBlockJoinQuery; import org.apache.lucene.search.join.ScoreMode; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.action.search.MaxScoreCollector; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.ParsingException; @@ -57,6 +58,7 @@ import java.util.Map; import java.util.Objects; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; import static org.elasticsearch.search.fetch.subphase.InnerHitsContext.intersect; public class NestedQueryBuilder extends AbstractQueryBuilder { @@ -266,6 +268,11 @@ protected int doHashCode() { @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[joining] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } + ObjectMapper nestedObjectMapper = context.getObjectMapper(path); if (nestedObjectMapper == null) { if (ignoreUnmapped) { diff --git a/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java b/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java index d25f98ee134b8..611a6fbe6d509 100644 --- a/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java +++ b/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java @@ -66,6 +66,7 @@ import java.util.Set; import java.util.function.BiConsumer; import java.util.function.BiFunction; +import java.util.function.BooleanSupplier; import java.util.function.LongSupplier; import java.util.function.Predicate; @@ -92,6 +93,7 @@ public class QueryShardContext extends QueryRewriteContext { private final Index fullyQualifiedIndex; private final Predicate indexNameMatcher; + private final BooleanSupplier allowExpensiveQueries; private final Map namedQueries = new HashMap<>(); private boolean allowUnmappedFields; @@ -112,18 +114,19 @@ public QueryShardContext(int shardId, IndexSearcher searcher, LongSupplier nowInMillis, String clusterAlias, - Predicate indexNameMatcher) { + Predicate indexNameMatcher, + BooleanSupplier allowExpensiveQueries) { this(shardId, indexSettings, bigArrays, bitsetFilterCache, indexFieldDataLookup, mapperService, similarityService, - scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, indexNameMatcher, - new Index(RemoteClusterAware.buildRemoteIndexName(clusterAlias, indexSettings.getIndex().getName()), - indexSettings.getIndex().getUUID())); + scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, indexNameMatcher, + new Index(RemoteClusterAware.buildRemoteIndexName(clusterAlias, indexSettings.getIndex().getName()), + indexSettings.getIndex().getUUID()), allowExpensiveQueries); } public QueryShardContext(QueryShardContext source) { this(source.shardId, source.indexSettings, source.bigArrays, source.bitsetFilterCache, source.indexFieldDataService, source.mapperService, source.similarityService, source.scriptService, source.getXContentRegistry(), source.getWriteableRegistry(), source.client, source.searcher, source.nowInMillis, source.indexNameMatcher, - source.fullyQualifiedIndex); + source.fullyQualifiedIndex, source.allowExpensiveQueries); } private QueryShardContext(int shardId, @@ -140,7 +143,8 @@ private QueryShardContext(int shardId, IndexSearcher searcher, LongSupplier nowInMillis, Predicate indexNameMatcher, - Index fullyQualifiedIndex) { + Index fullyQualifiedIndex, + BooleanSupplier allowExpensiveQueries) { super(xContentRegistry, namedWriteableRegistry, client, nowInMillis); this.shardId = shardId; this.similarityService = similarityService; @@ -155,6 +159,7 @@ private QueryShardContext(int shardId, this.searcher = searcher; this.indexNameMatcher = indexNameMatcher; this.fullyQualifiedIndex = fullyQualifiedIndex; + this.allowExpensiveQueries = allowExpensiveQueries; } private void reset() { @@ -192,6 +197,10 @@ public BitSetProducer bitsetFilter(Query filter) { return bitsetFilterCache.getBitSetProducer(filter); } + public boolean allowExpensiveQueries() { + return allowExpensiveQueries.getAsBoolean(); + } + @SuppressWarnings("unchecked") public > IFD getForField(MappedFieldType fieldType) { return (IFD) indexFieldDataService.apply(fieldType, fullyQualifiedIndex.getName()); diff --git a/server/src/main/java/org/elasticsearch/index/query/ScriptQueryBuilder.java b/server/src/main/java/org/elasticsearch/index/query/ScriptQueryBuilder.java index e9b18bd0aa1f1..da3f7850c0eb5 100644 --- a/server/src/main/java/org/elasticsearch/index/query/ScriptQueryBuilder.java +++ b/server/src/main/java/org/elasticsearch/index/query/ScriptQueryBuilder.java @@ -29,6 +29,7 @@ import org.apache.lucene.search.Scorer; import org.apache.lucene.search.TwoPhaseIterator; import org.apache.lucene.search.Weight; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.ParsingException; import org.elasticsearch.common.io.stream.StreamInput; import org.elasticsearch.common.io.stream.StreamOutput; @@ -40,6 +41,8 @@ import java.io.IOException; import java.util.Objects; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; + public class ScriptQueryBuilder extends AbstractQueryBuilder { public static final String NAME = "script"; @@ -130,6 +133,10 @@ public static ScriptQueryBuilder fromXContent(XContentParser parser) throws IOEx @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[script] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } FilterScript.Factory factory = context.compile(script, FilterScript.CONTEXT); FilterScript.LeafFactory filterScript = factory.newFactory(script.getParams(), context.lookup()); return new ScriptQuery(script, filterScript); diff --git a/server/src/main/java/org/elasticsearch/index/query/functionscore/ScriptScoreQueryBuilder.java b/server/src/main/java/org/elasticsearch/index/query/functionscore/ScriptScoreQueryBuilder.java index 74e51ff09a683..ff1db8a9e8c8f 100644 --- a/server/src/main/java/org/elasticsearch/index/query/functionscore/ScriptScoreQueryBuilder.java +++ b/server/src/main/java/org/elasticsearch/index/query/functionscore/ScriptScoreQueryBuilder.java @@ -20,6 +20,7 @@ package org.elasticsearch.index.query.functionscore; import org.apache.lucene.search.Query; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.Version; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.io.stream.StreamInput; @@ -42,6 +43,7 @@ import static org.elasticsearch.common.xcontent.ConstructingObjectParser.constructorArg; import static org.elasticsearch.common.xcontent.ConstructingObjectParser.optionalConstructorArg; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; /** * A query that computes a document score based on the provided script @@ -170,6 +172,10 @@ protected int doHashCode() { @Override protected Query doToQuery(QueryShardContext context) throws IOException { + if (context.allowExpensiveQueries() == false) { + throw new ElasticsearchException("[script score] queries cannot be executed when '" + + ALLOW_EXPENSIVE_QUERIES.getKey() + "' is set to false."); + } ScoreScript.Factory factory = context.compile(script, ScoreScript.CONTEXT); ScoreScript.LeafFactory scoreScriptFactory = factory.newFactory(script.getParams(), context.lookup()); Query query = this.query.toQuery(context); diff --git a/server/src/main/java/org/elasticsearch/index/search/MatchQuery.java b/server/src/main/java/org/elasticsearch/index/search/MatchQuery.java index d42466b5ebe66..abec51d0039f9 100644 --- a/server/src/main/java/org/elasticsearch/index/search/MatchQuery.java +++ b/server/src/main/java/org/elasticsearch/index/search/MatchQuery.java @@ -525,7 +525,8 @@ protected Query newTermQuery(Term term) { Supplier querySupplier; if (fuzziness != null) { querySupplier = () -> { - Query query = fieldType.fuzzyQuery(term.text(), fuzziness, fuzzyPrefixLength, maxExpansions, transpositions); + Query query = fieldType.fuzzyQuery(term.text(), fuzziness, fuzzyPrefixLength, maxExpansions, + transpositions, context); if (query instanceof FuzzyQuery) { QueryParsers.setRewriteMethod((FuzzyQuery) query, fuzzyRewriteMethod); } diff --git a/server/src/main/java/org/elasticsearch/index/search/QueryStringQueryParser.java b/server/src/main/java/org/elasticsearch/index/search/QueryStringQueryParser.java index a6beaf02d2e8c..a2527e040a3c1 100644 --- a/server/src/main/java/org/elasticsearch/index/search/QueryStringQueryParser.java +++ b/server/src/main/java/org/elasticsearch/index/search/QueryStringQueryParser.java @@ -463,7 +463,7 @@ private Query getFuzzyQuerySingle(String field, String termStr, int minSimilarit Analyzer normalizer = forceAnalyzer == null ? queryBuilder.context.getSearchAnalyzer(currentFieldType) : forceAnalyzer; BytesRef term = termStr == null ? null : normalizer.normalize(field, termStr); return currentFieldType.fuzzyQuery(term, Fuzziness.fromEdits(minSimilarity), - getFuzzyPrefixLength(), fuzzyMaxExpansions, fuzzyTranspositions); + getFuzzyPrefixLength(), fuzzyMaxExpansions, fuzzyTranspositions, context); } catch (RuntimeException e) { if (lenient) { return newLenientFieldQuery(field, e); diff --git a/server/src/main/java/org/elasticsearch/index/search/SimpleQueryStringQueryParser.java b/server/src/main/java/org/elasticsearch/index/search/SimpleQueryStringQueryParser.java index 912e03ca7996c..b8509ca2c112c 100644 --- a/server/src/main/java/org/elasticsearch/index/search/SimpleQueryStringQueryParser.java +++ b/server/src/main/java/org/elasticsearch/index/search/SimpleQueryStringQueryParser.java @@ -42,10 +42,10 @@ import org.elasticsearch.index.query.SimpleQueryStringBuilder; import java.io.IOException; +import java.util.ArrayList; +import java.util.List; import java.util.Map; import java.util.Objects; -import java.util.List; -import java.util.ArrayList; import static org.elasticsearch.common.lucene.search.Queries.newUnmappedFieldQuery; @@ -134,7 +134,7 @@ public Query newFuzzyQuery(String text, int fuzziness) { try { final BytesRef term = getAnalyzer(ft).normalize(fieldName, text); Query query = ft.fuzzyQuery(term, Fuzziness.fromEdits(fuzziness), settings.fuzzyPrefixLength, - settings.fuzzyMaxExpansions, settings.fuzzyTranspositions); + settings.fuzzyMaxExpansions, settings.fuzzyTranspositions, context); disjuncts.add(wrapWithBoost(query, entry.getValue())); } catch (RuntimeException e) { disjuncts.add(rethrowUnlessLenient(e)); diff --git a/server/src/main/java/org/elasticsearch/indices/IndicesService.java b/server/src/main/java/org/elasticsearch/indices/IndicesService.java index 62be919485669..2ba68d698c072 100644 --- a/server/src/main/java/org/elasticsearch/indices/IndicesService.java +++ b/server/src/main/java/org/elasticsearch/indices/IndicesService.java @@ -168,6 +168,7 @@ import static org.elasticsearch.index.IndexService.IndexCreationContext.CREATE_INDEX; import static org.elasticsearch.index.IndexService.IndexCreationContext.META_DATA_VERIFICATION; import static org.elasticsearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder; +import static org.elasticsearch.search.SearchService.ALLOW_EXPENSIVE_QUERIES; public class IndicesService extends AbstractLifecycleComponent implements IndicesClusterStateService.AllocatedIndices, IndexService.ShardStoreDeleter { @@ -220,6 +221,7 @@ public class IndicesService extends AbstractLifecycleComponent final AbstractRefCounted indicesRefCount; // pkg-private for testing private final CountDownLatch closeLatch = new CountDownLatch(1); private volatile boolean idFieldDataEnabled; + private volatile boolean allowExpensiveQueries; @Nullable private final EsThreadPoolExecutor danglingIndicesThreadPoolExecutor; @@ -316,6 +318,9 @@ protected void closeInternal() { 0, TimeUnit.MILLISECONDS, daemonThreadFactory(nodeName, DANGLING_INDICES_UPDATE_THREAD_NAME), threadPool.getThreadContext()) : null; + + this.allowExpensiveQueries = ALLOW_EXPENSIVE_QUERIES.get(clusterService.getSettings()); + clusterService.getClusterSettings().addSettingsUpdateConsumer(ALLOW_EXPENSIVE_QUERIES, this::setAllowExpensiveQueries); } private static final String DANGLING_INDICES_UPDATE_THREAD_NAME = "DanglingIndices#updateTask"; @@ -586,7 +591,8 @@ private synchronized IndexService createIndexService(IndexService.IndexCreationC idxSettings.getNumberOfReplicas(), indexCreationContext); - final IndexModule indexModule = new IndexModule(idxSettings, analysisRegistry, getEngineFactory(idxSettings), directoryFactories); + final IndexModule indexModule = new IndexModule(idxSettings, analysisRegistry, getEngineFactory(idxSettings), + directoryFactories, () -> allowExpensiveQueries); for (IndexingOperationListener operationListener : indexingOperationListeners) { indexModule.addIndexOperationListener(operationListener); } @@ -655,7 +661,8 @@ private EngineFactory getEngineFactory(final IndexSettings idxSettings) { */ public synchronized MapperService createIndexMapperService(IndexMetaData indexMetaData) throws IOException { final IndexSettings idxSettings = new IndexSettings(indexMetaData, this.settings, indexScopedSettings); - final IndexModule indexModule = new IndexModule(idxSettings, analysisRegistry, getEngineFactory(idxSettings), directoryFactories); + final IndexModule indexModule = new IndexModule(idxSettings, analysisRegistry, getEngineFactory(idxSettings), + directoryFactories, () -> allowExpensiveQueries); pluginsService.onIndexModule(indexModule); return indexModule.newIndexMapperService(xContentRegistry, mapperRegistry, scriptService); } @@ -1572,6 +1579,10 @@ protected void doRun() { } } + private void setAllowExpensiveQueries(Boolean allowExpensiveQueries) { + this.allowExpensiveQueries = allowExpensiveQueries; + } + // visible for testing public boolean allPendingDanglingIndicesWritten() { return nodeWriteDanglingIndicesInfo == false || diff --git a/server/src/main/java/org/elasticsearch/search/SearchService.java b/server/src/main/java/org/elasticsearch/search/SearchService.java index d3afcd5c4677a..412f1a3f11598 100644 --- a/server/src/main/java/org/elasticsearch/search/SearchService.java +++ b/server/src/main/java/org/elasticsearch/search/SearchService.java @@ -136,6 +136,8 @@ public class SearchService extends AbstractLifecycleComponent implements IndexEv Setting.positiveTimeSetting("search.max_keep_alive", timeValueHours(24), Property.NodeScope, Property.Dynamic); public static final Setting KEEPALIVE_INTERVAL_SETTING = Setting.positiveTimeSetting("search.keep_alive_interval", timeValueMinutes(1), Property.NodeScope); + public static final Setting ALLOW_EXPENSIVE_QUERIES = + Setting.boolSetting("search.allow_expensive_queries", true, Property.NodeScope, Property.Dynamic); /** * Enables low-level, frequent search cancellation checks. Enabling low-level checks will make long running searches to react diff --git a/server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataCreateIndexServiceTests.java b/server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataCreateIndexServiceTests.java index 66843e1fb5e9b..9a955146d18df 100644 --- a/server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataCreateIndexServiceTests.java +++ b/server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataCreateIndexServiceTests.java @@ -128,7 +128,7 @@ public void setupCreateIndexRequestAndAliasValidator() { queryShardContext = new QueryShardContext(0, new IndexSettings(IndexMetaData.builder("test").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> randomNonNegativeLong(), null, null); + null, null, () -> randomNonNegativeLong(), null, null, () -> true); } private ClusterState createClusterState(String name, int numShards, int numReplicas, Settings settings) { diff --git a/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java b/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java index 2c11b26580f3d..7807f58c7fd6e 100644 --- a/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java +++ b/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java @@ -166,7 +166,8 @@ private IndexService newIndexService(IndexModule module) throws IOException { public void testWrapperIsBound() throws IOException { final MockEngineFactory engineFactory = new MockEngineFactory(AssertingDirectoryReader.class); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, engineFactory, Collections.emptyMap()); + IndexModule module = new IndexModule( + indexSettings, emptyAnalysisRegistry, engineFactory, Collections.emptyMap(), () -> true); module.setReaderWrapper(s -> new Wrapper()); IndexService indexService = newIndexService(module); @@ -186,7 +187,8 @@ public void testRegisterIndexStore() throws IOException { final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings); final Map indexStoreFactories = singletonMap( "foo_store", new FooFunction()); - final IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), indexStoreFactories); + final IndexModule module = new IndexModule( + indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), indexStoreFactories, () -> true); final IndexService indexService = newIndexService(module); assertThat(indexService.getDirectoryFactory(), instanceOf(FooFunction.class)); @@ -203,7 +205,7 @@ public void beforeIndexRemoved(IndexService indexService, IndexRemovalReason rea } }; IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); module.addIndexEventListener(eventListener); IndexService indexService = newIndexService(module); IndexSettings x = indexService.getIndexSettings(); @@ -218,7 +220,7 @@ public void beforeIndexRemoved(IndexService indexService, IndexRemovalReason rea public void testListener() throws IOException { Setting booleanSetting = Setting.boolSetting("index.foo.bar", false, Property.Dynamic, Property.IndexScope); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings, booleanSetting); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); Setting booleanSetting2 = Setting.boolSetting("index.foo.bar.baz", false, Property.Dynamic, Property.IndexScope); AtomicBoolean atomicBoolean = new AtomicBoolean(false); module.addSettingsUpdateConsumer(booleanSetting, atomicBoolean::set); @@ -238,7 +240,7 @@ public void testListener() throws IOException { public void testAddIndexOperationListener() throws IOException { final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); AtomicBoolean executed = new AtomicBoolean(false); IndexingOperationListener listener = new IndexingOperationListener() { @Override @@ -269,7 +271,7 @@ public Engine.Index preIndex(ShardId shardId, Engine.Index operation) { public void testAddSearchOperationListener() throws IOException { final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); AtomicBoolean executed = new AtomicBoolean(false); SearchOperationListener listener = new SearchOperationListener() { @@ -304,7 +306,7 @@ public void testAddSimilarity() throws IOException { .build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); IndexModule module = - new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + createIndexModule(indexSettings, emptyAnalysisRegistry); module.addSimilarity("test_similarity", (providerSettings, indexCreatedVersion, scriptService) -> new TestSimilarity(providerSettings.get("key"))); @@ -320,9 +322,11 @@ public void testAddSimilarity() throws IOException { indexService.close("simon says", false); } + + public void testFrozen() { final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(index, settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); module.freeze(); String msg = "Can't modify IndexModule once the index service has been created"; assertEquals(msg, expectThrows(IllegalStateException.class, () -> module.addSearchOperationListener(null)).getMessage()); @@ -333,7 +337,7 @@ public void testFrozen() { assertEquals(msg, expectThrows(IllegalStateException.class, () -> module.forceQueryCacheProvider(null)).getMessage()); } - public void testSetupUnknownSimilarity() throws IOException { + public void testSetupUnknownSimilarity() { Settings settings = Settings.builder() .put("index.similarity.my_similarity.type", "test_similarity") .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT) @@ -341,19 +345,19 @@ public void testSetupUnknownSimilarity() throws IOException { .build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); IndexModule module = - new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + createIndexModule(indexSettings, emptyAnalysisRegistry); Exception ex = expectThrows(IllegalArgumentException.class, () -> newIndexService(module)); assertEquals("Unknown Similarity type [test_similarity] for [my_similarity]", ex.getMessage()); } - public void testSetupWithoutType() throws IOException { + public void testSetupWithoutType() { Settings settings = Settings.builder() .put("index.similarity.my_similarity.foo", "bar") .put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()) .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT) .build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); Exception ex = expectThrows(IllegalArgumentException.class, () -> newIndexService(module)); assertEquals("Similarity [my_similarity] must have an associated type", ex.getMessage()); } @@ -363,7 +367,7 @@ public void testForceCustomQueryCache() throws IOException { .put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()) .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); final Set liveQueryCaches = new HashSet<>(); module.forceQueryCacheProvider((a, b) -> { final CustomQueryCache customQueryCache = new CustomQueryCache(liveQueryCaches); @@ -384,7 +388,7 @@ public void testDefaultQueryCacheImplIsSelected() throws IOException { .put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()) .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); IndexService indexService = newIndexService(module); assertTrue(indexService.cache().query() instanceof IndexQueryCache); indexService.close("simon says", false); @@ -396,7 +400,7 @@ public void testDisableQueryCacheHasPrecedenceOverForceQueryCache() throws IOExc .put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()) .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); module.forceQueryCacheProvider((a, b) -> new CustomQueryCache(null)); IndexService indexService = newIndexService(module); assertTrue(indexService.cache().query() instanceof DisabledQueryCache); @@ -408,7 +412,7 @@ public void testCustomQueryCacheCleanedUpIfIndexServiceCreationFails() { .put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()) .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings("foo", settings); - IndexModule module = new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, emptyAnalysisRegistry); final Set liveQueryCaches = new HashSet<>(); module.forceQueryCacheProvider((a, b) -> { final CustomQueryCache customQueryCache = new CustomQueryCache(liveQueryCaches); @@ -458,7 +462,7 @@ public void close() { }; final AnalysisRegistry analysisRegistry = new AnalysisRegistry(environment, emptyMap(), emptyMap(), emptyMap(), singletonMap("test", analysisProvider), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap()); - IndexModule module = new IndexModule(indexSettings, analysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule module = createIndexModule(indexSettings, analysisRegistry); threadPool.shutdown(); // causes index service creation to fail expectThrows(EsRejectedExecutionException.class, () -> newIndexService(module)); assertThat(openAnalyzers, empty()); @@ -475,11 +479,16 @@ public void testMmapNotAllowed() { .build(); final IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(new Index("foo", "_na_"), settings, nodeSettings); final IndexModule module = - new IndexModule(indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap()); + createIndexModule(indexSettings, emptyAnalysisRegistry); final IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> newIndexService(module)); assertThat(e, hasToString(containsString("store type [" + storeType + "] is not allowed"))); } + private static IndexModule createIndexModule(IndexSettings indexSettings, AnalysisRegistry emptyAnalysisRegistry) { + return new IndexModule( + indexSettings, emptyAnalysisRegistry, new InternalEngineFactory(), Collections.emptyMap(), () -> true); + } + class CustomQueryCache implements QueryCache { private final Set liveQueryCaches; @@ -545,5 +554,4 @@ public DirectoryReader apply(DirectoryReader reader) { return null; } } - } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java index d18d0fb097746..011d5ae3ef6c1 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java @@ -231,7 +231,7 @@ public void testTermQuery() { QueryShardContext context = new QueryShardContext(0, new IndexSettings(IndexMetaData.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null); + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true); MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); String date = "2015-10-12T14:10:55"; @@ -254,7 +254,7 @@ public void testRangeQuery() throws IOException { QueryShardContext context = new QueryShardContext(0, new IndexSettings(IndexMetaData.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> nowInMillis, null, null); + null, null, () -> nowInMillis, null, null, () -> true); MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); String date1 = "2015-10-12T14:10:55"; diff --git a/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java index dbff39027f5d6..4635c6f28ccf7 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java @@ -68,7 +68,7 @@ public void testTermQuery() { QueryShardContext queryShardContext = new QueryShardContext(0, indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, - null, null, null, null, null, null, () -> 0L, null, null); + null, null, null, null, null, null, () -> 0L, null, null, () -> true); fieldNamesFieldType.setEnabled(true); Query termQuery = fieldNamesFieldType.termQuery("field_name", queryShardContext); assertEquals(new TermQuery(new Term(FieldNamesFieldMapper.CONTENT_TYPE, "field_name")), termQuery); diff --git a/server/src/test/java/org/elasticsearch/index/mapper/IgnoredFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/IgnoredFieldTypeTests.java index e0cd3b1d153fd..662b2e331e49e 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/IgnoredFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/IgnoredFieldTypeTests.java @@ -26,6 +26,7 @@ import org.apache.lucene.search.RegexpQuery; import org.apache.lucene.search.WildcardQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; public class IgnoredFieldTypeTests extends FieldTypeTestCase { @@ -40,7 +41,12 @@ public void testPrefixQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new PrefixQuery(new Term("field", new BytesRef("foo*"))); - assertEquals(expected, ft.prefixQuery("foo*", null, null)); + assertEquals(expected, ft.prefixQuery("foo*", null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.prefixQuery("foo*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[prefix] queries cannot be executed when 'search.allow_expensive_queries' is set to false. " + + "For optimised prefix queries on text fields please enable [index_prefixes].", ee.getMessage()); } public void testRegexpQuery() { @@ -49,7 +55,12 @@ public void testRegexpQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new RegexpQuery(new Term("field", new BytesRef("foo?"))); - assertEquals(expected, ft.regexpQuery("foo?", 0, 10, null, null)); + assertEquals(expected, ft.regexpQuery("foo?", 0, 10, null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.regexpQuery("foo?", randomInt(10), randomInt(10) + 1, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[regexp] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testWildcardQuery() { @@ -58,6 +69,11 @@ public void testWildcardQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new WildcardQuery(new Term("field", new BytesRef("foo*"))); - assertEquals(expected, ft.wildcardQuery("foo*", null, null)); + assertEquals(expected, ft.wildcardQuery("foo*", null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.wildcardQuery("valu*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[wildcard] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java index 1b100fb0872ea..bf0a0dffba743 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java @@ -78,6 +78,6 @@ private QueryShardContext createContext() { Predicate indexNameMatcher = pattern -> Regex.simpleMatch(pattern, "index"); return new QueryShardContext(0, indexSettings, null, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, System::currentTimeMillis, null, indexNameMatcher); + null, null, System::currentTimeMillis, null, indexNameMatcher, () -> true); } } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/KeywordFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/KeywordFieldTypeTests.java index eae5b4ac7d2ab..b05c4c779cf6e 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/KeywordFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/KeywordFieldTypeTests.java @@ -33,7 +33,10 @@ import org.apache.lucene.search.RegexpQuery; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; +import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.lucene.BytesRefs; import org.elasticsearch.common.lucene.Lucene; import org.elasticsearch.common.unit.Fuzziness; import org.elasticsearch.index.analysis.AnalyzerScope; @@ -150,17 +153,35 @@ public void testExistsQuery() { assertEquals(new TermQuery(new Term(FieldNamesFieldMapper.NAME, "field")), ft.existsQuery(null)); } + public void testRangeQuery() { + MappedFieldType ft = createDefaultFieldType(); + ft.setName("field"); + ft.setIndexOptions(IndexOptions.DOCS); + assertEquals(new TermRangeQuery("field", BytesRefs.toBytesRef("foo"), BytesRefs.toBytesRef("bar"), true, false), + ft.rangeQuery("foo", "bar", true, false, null, null, null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.rangeQuery("foo", "bar", true, false, null, null, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[range] queries on [text] or [keyword] fields cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", ee.getMessage()); + } + public void testRegexpQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); assertEquals(new RegexpQuery(new Term("field","foo.*")), - ft.regexpQuery("foo.*", 0, 10, null, null)); + ft.regexpQuery("foo.*", 0, 10, null, MOCK_QSC)); ft.setIndexOptions(IndexOptions.NONE); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, - () -> ft.regexpQuery("foo.*", 0, 10, null, null)); + () -> ft.regexpQuery("foo.*", 0, 10, null, MOCK_QSC)); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.regexpQuery("foo.*", randomInt(10), randomInt(10) + 1, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[regexp] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testFuzzyQuery() { @@ -168,12 +189,18 @@ public void testFuzzyQuery() { ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); assertEquals(new FuzzyQuery(new Term("field","foo"), 2, 1, 50, true), - ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true)); + ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC)); ft.setIndexOptions(IndexOptions.NONE); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true)); + () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC)); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.fuzzyQuery("foo", Fuzziness.AUTO, randomInt(10) + 1, randomInt(10) + 1, + randomBoolean(), MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[fuzzy] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testNormalizeQueries() { diff --git a/server/src/test/java/org/elasticsearch/index/mapper/LegacyGeoShapeFieldMapperTests.java b/server/src/test/java/org/elasticsearch/index/mapper/LegacyGeoShapeFieldMapperTests.java index d53eb0bcac134..aaabf3f9edbf0 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/LegacyGeoShapeFieldMapperTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/LegacyGeoShapeFieldMapperTests.java @@ -22,15 +22,20 @@ import org.apache.lucene.spatial.prefix.RecursivePrefixTreeStrategy; import org.apache.lucene.spatial.prefix.tree.GeohashPrefixTree; import org.apache.lucene.spatial.prefix.tree.QuadPrefixTree; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.ElasticsearchParseException; import org.elasticsearch.common.Explicit; import org.elasticsearch.common.Strings; import org.elasticsearch.common.compress.CompressedXContent; import org.elasticsearch.common.geo.GeoUtils; +import org.elasticsearch.common.geo.ShapeRelation; +import org.elasticsearch.common.geo.SpatialStrategy; import org.elasticsearch.common.geo.builders.ShapeBuilder; import org.elasticsearch.common.xcontent.ToXContent; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentFactory; +import org.elasticsearch.geometry.Point; +import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.plugins.Plugin; import org.elasticsearch.test.ESSingleNodeTestCase; import org.elasticsearch.test.InternalSettingsPlugin; @@ -44,6 +49,8 @@ import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.instanceOf; import static org.hamcrest.Matchers.not; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class LegacyGeoShapeFieldMapperTests extends ESSingleNodeTestCase { @@ -695,6 +702,31 @@ public void testPointsOnlyFalseWithTermStrategy() throws Exception { assertFieldWarnings("tree", "precision", "strategy", "points_only"); } + public void testDisallowExpensiveQueries() throws IOException { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + String mapping = Strings.toString(XContentFactory.jsonBuilder().startObject().startObject("type1") + .startObject("properties").startObject("location") + .field("type", "geo_shape") + .field("tree", "quadtree") + .endObject().endObject() + .endObject().endObject()); + + DocumentMapper defaultMapper = createIndex("test").mapperService().documentMapperParser() + .parse("type1", new CompressedXContent(mapping)); + Mapper fieldMapper = defaultMapper.mappers().getMapper("location"); + assertThat(fieldMapper, instanceOf(LegacyGeoShapeFieldMapper.class)); + LegacyGeoShapeFieldMapper geoShapeFieldMapper = (LegacyGeoShapeFieldMapper) fieldMapper; + + + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> geoShapeFieldMapper.fieldType().geometryQueryBuilder().process( + new Point(-10, 10), "location", SpatialStrategy.TERM, ShapeRelation.INTERSECTS, queryShardContext)); + assertEquals("[geo-shape] queries on [PrefixTree geo shapes] cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", e.getMessage()); + assertFieldWarnings("tree"); + } + public String toXContentString(LegacyGeoShapeFieldMapper mapper, boolean includeDefaults) throws IOException { XContentBuilder builder = XContentFactory.jsonBuilder().startObject(); ToXContent.Params params; @@ -710,5 +742,4 @@ public String toXContentString(LegacyGeoShapeFieldMapper mapper, boolean include public String toXContentString(LegacyGeoShapeFieldMapper mapper) throws IOException { return toXContentString(mapper, true); } - } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java index 893a909ece2c8..d1a7ff06d3276 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java @@ -230,7 +230,7 @@ private QueryShardContext createContext() { .put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).build(); IndexSettings idxSettings = IndexSettingsModule.newIndexSettings(randomAlphaOfLengthBetween(1, 10), indexSettings); return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null); + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true); } public void testDateRangeQueryUsingMappingFormat() { diff --git a/server/src/test/java/org/elasticsearch/index/mapper/RoutingFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/RoutingFieldTypeTests.java index 6f68d28c0176a..8bb052efbd467 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/RoutingFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/RoutingFieldTypeTests.java @@ -25,8 +25,10 @@ import org.apache.lucene.search.RegexpQuery; import org.apache.lucene.search.WildcardQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; public class RoutingFieldTypeTests extends FieldTypeTestCase { + @Override protected MappedFieldType createDefaultFieldType() { return new RoutingFieldMapper.RoutingFieldType(); @@ -38,7 +40,12 @@ public void testPrefixQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new PrefixQuery(new Term("field", new BytesRef("foo*"))); - assertEquals(expected, ft.prefixQuery("foo*", null, null)); + assertEquals(expected, ft.prefixQuery("foo*", null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.prefixQuery("foo*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[prefix] queries cannot be executed when 'search.allow_expensive_queries' is set to false. " + + "For optimised prefix queries on text fields please enable [index_prefixes].", ee.getMessage()); } public void testRegexpQuery() { @@ -47,7 +54,12 @@ public void testRegexpQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new RegexpQuery(new Term("field", new BytesRef("foo?"))); - assertEquals(expected, ft.regexpQuery("foo?", 0, 10, null, null)); + assertEquals(expected, ft.regexpQuery("foo?", 0, 10, null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.regexpQuery("foo?", randomInt(10), randomInt(10) + 1, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[regexp] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testWildcardQuery() { @@ -56,6 +68,11 @@ public void testWildcardQuery() { ft.setIndexOptions(IndexOptions.DOCS); Query expected = new WildcardQuery(new Term("field", new BytesRef("foo*"))); - assertEquals(expected, ft.wildcardQuery("foo*", null, null)); + assertEquals(expected, ft.wildcardQuery("foo*", null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.wildcardQuery("valu*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[wildcard] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/TextFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/TextFieldTypeTests.java index da58907355202..9b54dcda3100a 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/TextFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/TextFieldTypeTests.java @@ -30,10 +30,13 @@ import org.apache.lucene.search.RegexpQuery; import org.apache.lucene.search.TermInSetQuery; import org.apache.lucene.search.TermQuery; +import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.automaton.Automata; import org.apache.lucene.util.automaton.Automaton; import org.apache.lucene.util.automaton.Operations; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.lucene.BytesRefs; import org.elasticsearch.common.unit.Fuzziness; import org.junit.Before; @@ -45,6 +48,7 @@ import static org.hamcrest.Matchers.equalTo; public class TextFieldTypeTests extends FieldTypeTestCase { + @Override protected MappedFieldType createDefaultFieldType() { return new TextFieldMapper.TextFieldType(); @@ -130,17 +134,35 @@ public void testTermsQuery() { assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); } + public void testRangeQuery() { + MappedFieldType ft = createDefaultFieldType(); + ft.setName("field"); + ft.setIndexOptions(IndexOptions.DOCS); + assertEquals(new TermRangeQuery("field", BytesRefs.toBytesRef("foo"), BytesRefs.toBytesRef("bar"), true, false), + ft.rangeQuery("foo", "bar", true, false, null, null, null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.rangeQuery("foo", "bar", true, false, null, null, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[range] queries on [text] or [keyword] fields cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", ee.getMessage()); + } + public void testRegexpQuery() { MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); assertEquals(new RegexpQuery(new Term("field","foo.*")), - ft.regexpQuery("foo.*", 0, 10, null, null)); + ft.regexpQuery("foo.*", 0, 10, null, MOCK_QSC)); ft.setIndexOptions(IndexOptions.NONE); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, - () -> ft.regexpQuery("foo.*", 0, 10, null, null)); + () -> ft.regexpQuery("foo.*", 0, 10, null, MOCK_QSC)); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.regexpQuery("foo.*", randomInt(10), randomInt(10) + 1, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[regexp] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testFuzzyQuery() { @@ -148,12 +170,18 @@ public void testFuzzyQuery() { ft.setName("field"); ft.setIndexOptions(IndexOptions.DOCS); assertEquals(new FuzzyQuery(new Term("field","foo"), 2, 1, 50, true), - ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true)); + ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC)); ft.setIndexOptions(IndexOptions.NONE); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, - () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true)); + () -> ft.fuzzyQuery("foo", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC)); assertEquals("Cannot search on field [field] since it is not indexed.", e.getMessage()); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.fuzzyQuery("foo", Fuzziness.AUTO, randomInt(10) + 1, randomInt(10) + 1, + randomBoolean(), MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[fuzzy] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testIndexPrefixes() { @@ -161,13 +189,18 @@ public void testIndexPrefixes() { ft.setName("field"); ft.setPrefixFieldType(new TextFieldMapper.PrefixFieldType("field", "field._index_prefix", 2, 10)); - Query q = ft.prefixQuery("goin", CONSTANT_SCORE_REWRITE, null); + Query q = ft.prefixQuery("goin", CONSTANT_SCORE_REWRITE, randomMockShardContext()); assertEquals(new ConstantScoreQuery(new TermQuery(new Term("field._index_prefix", "goin"))), q); - q = ft.prefixQuery("internationalisatio", CONSTANT_SCORE_REWRITE, null); + q = ft.prefixQuery("internationalisatio", CONSTANT_SCORE_REWRITE, MOCK_QSC); assertEquals(new PrefixQuery(new Term("field", "internationalisatio")), q); - q = ft.prefixQuery("g", CONSTANT_SCORE_REWRITE, null); + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.prefixQuery("internationalisatio", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[prefix] queries cannot be executed when 'search.allow_expensive_queries' is set to false. " + + "For optimised prefix queries on text fields please enable [index_prefixes].", ee.getMessage()); + + q = ft.prefixQuery("g", CONSTANT_SCORE_REWRITE, randomMockShardContext()); Automaton automaton = Operations.concatenate(Arrays.asList(Automata.makeChar('g'), Automata.makeAnyChar())); diff --git a/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java b/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java index 1e147e24def7a..8e736b12d6a1f 100644 --- a/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java @@ -425,7 +425,7 @@ public FactoryType compile(Script script, ScriptContext true); String json = "{ \"intervals\" : { \"" + STRING_FIELD_NAME + "\": { " + "\"match\" : { " + diff --git a/server/src/test/java/org/elasticsearch/index/query/NestedQueryBuilderTests.java b/server/src/test/java/org/elasticsearch/index/query/NestedQueryBuilderTests.java index 4f4aff6d10cdf..c7f703113f264 100644 --- a/server/src/test/java/org/elasticsearch/index/query/NestedQueryBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/NestedQueryBuilderTests.java @@ -23,6 +23,7 @@ import org.apache.lucene.search.MatchNoDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.join.ScoreMode; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.Version; import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest; import org.elasticsearch.common.Strings; @@ -354,5 +355,17 @@ public void testExtractInnerHitBuildersWithDuplicate() { queryBuilder.innerHit(new InnerHitBuilder("some_name")); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> InnerHitContextBuilder.extractInnerHits(queryBuilder,Collections.singletonMap("some_name", null))); + assertEquals("[inner_hits] already contains an entry for key [some_name]", e.getMessage()); + } + + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + NestedQueryBuilder queryBuilder = new NestedQueryBuilder("path", new MatchAllQueryBuilder(), ScoreMode.None); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[joining] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); } } diff --git a/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java b/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java index 8e57ad50bd1cd..ba68e1b6a2013 100644 --- a/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java @@ -153,6 +153,6 @@ public static QueryShardContext createQueryShardContext(String indexUuid, String (mappedFieldType, idxName) -> mappedFieldType.fielddataBuilder(idxName).build(indexSettings, mappedFieldType, null, null, null), mapperService, null, null, NamedXContentRegistry.EMPTY, new NamedWriteableRegistry(Collections.emptyList()), - null, null, () -> nowInMillis, clusterAlias, null); + null, null, () -> nowInMillis, clusterAlias, null, () -> true); } } diff --git a/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java b/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java index 83ab9c8e62bb4..c43470ea3b21c 100644 --- a/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java @@ -41,7 +41,7 @@ public void testRewriteMissingField() throws Exception { IndexReader reader = new MultiReader(); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), BigArrays.NON_RECYCLING_INSTANCE, null, null, indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), - null, new IndexSearcher(reader), null, null, null); + null, new IndexSearcher(reader), null, null, null, () -> true); RangeQueryBuilder range = new RangeQueryBuilder("foo"); assertEquals(Relation.DISJOINT, range.getRelation(context)); } @@ -58,7 +58,8 @@ public void testRewriteMissingReader() throws Exception { indexService.mapperService().merge("type", new CompressedXContent(mapping), MergeReason.MAPPING_UPDATE); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), null, null, null, - indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), null, null, null, null, null); + indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), + null, null, null, null, null, () -> true); RangeQueryBuilder range = new RangeQueryBuilder("foo"); // can't make assumptions on a missing reader, so it must return INTERSECT assertEquals(Relation.INTERSECTS, range.getRelation(context)); @@ -78,7 +79,7 @@ public void testRewriteEmptyReader() throws Exception { IndexReader reader = new MultiReader(); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), BigArrays.NON_RECYCLING_INSTANCE, null, null, indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), - null, new IndexSearcher(reader), null, null, null); + null, new IndexSearcher(reader), null, null, null, () -> true); RangeQueryBuilder range = new RangeQueryBuilder("foo"); // no values -> DISJOINT assertEquals(Relation.DISJOINT, range.getRelation(context)); diff --git a/server/src/test/java/org/elasticsearch/index/query/ScriptQueryBuilderTests.java b/server/src/test/java/org/elasticsearch/index/query/ScriptQueryBuilderTests.java index fbf67860a2d85..7eda1083b263e 100644 --- a/server/src/test/java/org/elasticsearch/index/query/ScriptQueryBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/ScriptQueryBuilderTests.java @@ -20,6 +20,7 @@ package org.elasticsearch.index.query; import org.apache.lucene.search.Query; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.ParsingException; import org.elasticsearch.script.MockScriptEngine; import org.elasticsearch.script.Script; @@ -33,6 +34,8 @@ import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.instanceOf; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class ScriptQueryBuilderTests extends AbstractQueryTestCase { @Override @@ -53,7 +56,9 @@ protected void doAssertLuceneQuery(ScriptQueryBuilder queryBuilder, Query query, } public void testIllegalConstructorArg() { - expectThrows(IllegalArgumentException.class, () -> new ScriptQueryBuilder((Script) null)); + IllegalArgumentException e = expectThrows(IllegalArgumentException.class, + () -> new ScriptQueryBuilder((Script) null)); + assertEquals("script cannot be null", e.getMessage()); } public void testFromJsonVerbose() throws IOException { @@ -126,4 +131,15 @@ public void testCacheability() throws IOException { assertNotNull(rewriteQuery.toQuery(context)); assertFalse("query should not be cacheable: " + queryBuilder.toString(), context.isCacheable()); } + + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + ScriptQueryBuilder queryBuilder = doCreateTestQueryBuilder(); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[script] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); + } } diff --git a/server/src/test/java/org/elasticsearch/index/query/ScriptScoreQueryBuilderTests.java b/server/src/test/java/org/elasticsearch/index/query/ScriptScoreQueryBuilderTests.java index 690a7d6ae757a..04322a01d0f68 100644 --- a/server/src/test/java/org/elasticsearch/index/query/ScriptScoreQueryBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/ScriptScoreQueryBuilderTests.java @@ -20,6 +20,7 @@ package org.elasticsearch.index.query; import org.apache.lucene.search.Query; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.lucene.search.function.ScriptScoreQuery; import org.elasticsearch.index.query.functionscore.ScriptScoreQueryBuilder; import org.elasticsearch.script.MockScriptEngine; @@ -32,6 +33,8 @@ import static org.elasticsearch.index.query.QueryBuilders.matchAllQuery; import static org.hamcrest.CoreMatchers.instanceOf; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; public class ScriptScoreQueryBuilderTests extends AbstractQueryTestCase { @@ -71,15 +74,17 @@ public void testIllegalArguments() { String scriptStr = "1"; Script script = new Script(ScriptType.INLINE, MockScriptEngine.NAME, scriptStr, Collections.emptyMap()); - expectThrows( + IllegalArgumentException e = expectThrows( IllegalArgumentException.class, () -> new ScriptScoreQueryBuilder(matchAllQuery(), null) ); + assertEquals("script_score: script must not be null" , e.getMessage()); - expectThrows( + e = expectThrows( IllegalArgumentException.class, () -> new ScriptScoreQueryBuilder(null, script) ); + assertEquals("script_score: query must not be null" , e.getMessage()); } /** @@ -93,4 +98,15 @@ public void testCacheability() throws IOException { assertNotNull(rewriteQuery.toQuery(context)); assertFalse("query should not be cacheable: " + queryBuilder.toString(), context.isCacheable()); } + + public void testDisallowExpensiveQueries() { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(false); + + ScriptScoreQueryBuilder queryBuilder = doCreateTestQueryBuilder(); + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> queryBuilder.toQuery(queryShardContext)); + assertEquals("[script score] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getMessage()); + } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java index 6fc5561f6d6f7..4afce4e5ff247 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java @@ -97,7 +97,7 @@ public void testParseAndValidate() { QueryShardContext qsc = new QueryShardContext(0, new IndexSettings(IndexMetaData.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> now, null, null); + null, null, () -> now, null, null, () -> true); DateFormatter formatter = DateFormatter.forPattern("dateOptionalTime"); DocValueFormat format = new DocValueFormat.DateTime(formatter, ZoneOffset.UTC, DateFieldMapper.Resolution.MILLISECONDS); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java index 9d0d1d69f023c..4adf16f6012b9 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java @@ -426,6 +426,6 @@ protected QueryShardContext queryShardContextMock(IndexSearcher searcher, Map engines = Collections.singletonMap(scriptEngine.getType(), scriptEngine); ScriptService scriptService = new ScriptService(Settings.EMPTY, engines, ScriptModule.CORE_CONTEXTS); return new QueryShardContext(0, indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, scriptService, - xContentRegistry(), writableRegistry(), null, null, System::currentTimeMillis, null, null); + xContentRegistry(), writableRegistry(), null, null, System::currentTimeMillis, null, null, () -> true); } } diff --git a/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java b/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java index 9a542fd762ad9..31a2de6b3d821 100644 --- a/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java @@ -280,7 +280,7 @@ public void testBuildSearchContextHighlight() throws IOException { // shard context will only need indicesQueriesRegistry for building Query objects nested in highlighter QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), namedWriteableRegistry, - null, null, System::currentTimeMillis, null, null) { + null, null, System::currentTimeMillis, null, null, () -> true) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); diff --git a/server/src/test/java/org/elasticsearch/search/geo/GeoShapeIntegrationIT.java b/server/src/test/java/org/elasticsearch/search/geo/GeoShapeIntegrationIT.java index 83d94886fbb16..1ecc508fb3479 100644 --- a/server/src/test/java/org/elasticsearch/search/geo/GeoShapeIntegrationIT.java +++ b/server/src/test/java/org/elasticsearch/search/geo/GeoShapeIntegrationIT.java @@ -18,12 +18,14 @@ */ package org.elasticsearch.search.geo; +import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.cluster.ClusterState; import org.elasticsearch.cluster.routing.IndexShardRoutingTable; import org.elasticsearch.common.Strings; import org.elasticsearch.common.geo.builders.PointBuilder; import org.elasticsearch.common.geo.builders.ShapeBuilder; +import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentFactory; import org.elasticsearch.common.xcontent.XContentType; import org.elasticsearch.index.IndexService; @@ -41,6 +43,16 @@ public class GeoShapeIntegrationIT extends ESIntegTestCase { + @Override + protected Settings nodeSettings(int nodeOrdinal) { + return Settings.builder() + // Check that only geo-shape queries on legacy PrefixTree based + // geo shapes are disallowed. + .put("search.allow_expensive_queries", false) + .put(super.nodeSettings(nodeOrdinal)) + .build(); + } + /** * Test that orientation parameter correctly persists across cluster restart */ @@ -183,21 +195,21 @@ public void testIndexShapeRouting() throws Exception { public void testIndexPolygonDateLine() throws Exception { String mappingVector = "{\n" + - " \"properties\": {\n" + - " \"shape\": {\n" + - " \"type\": \"geo_shape\"\n" + - " }\n" + - " }\n" + - " }"; + " \"properties\": {\n" + + " \"shape\": {\n" + + " \"type\": \"geo_shape\"\n" + + " }\n" + + " }\n" + + " }"; String mappingQuad = "{\n" + - " \"properties\": {\n" + - " \"shape\": {\n" + - " \"type\": \"geo_shape\",\n" + - " \"tree\": \"quadtree\"\n" + - " }\n" + - " }\n" + - " }"; + " \"properties\": {\n" + + " \"shape\": {\n" + + " \"type\": \"geo_shape\",\n" + + " \"tree\": \"quadtree\"\n" + + " }\n" + + " }\n" + + " }"; // create index @@ -208,37 +220,47 @@ public void testIndexPolygonDateLine() throws Exception { ensureGreen(); String source = "{\n" + - " \"shape\" : \"POLYGON((179 0, -179 0, -179 2, 179 2, 179 0))\""+ - "}"; + " \"shape\" : \"POLYGON((179 0, -179 0, -179 2, 179 2, 179 0))\""+ + "}"; indexRandom(true, client().prepareIndex("quad").setId("0").setSource(source, XContentType.JSON)); indexRandom(true, client().prepareIndex("vector").setId("0").setSource(source, XContentType.JSON)); - SearchResponse searchResponse = client().prepareSearch("quad").setQuery( - geoShapeQuery("shape", new PointBuilder(-179.75, 1)) - ).get(); + try { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + SearchResponse searchResponse = client().prepareSearch("quad").setQuery( + geoShapeQuery("shape", new PointBuilder(-179.75, 1)) + ).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); - searchResponse = client().prepareSearch("quad").setQuery( - geoShapeQuery("shape", new PointBuilder(90, 1)) - ).get(); + searchResponse = client().prepareSearch("quad").setQuery( + geoShapeQuery("shape", new PointBuilder(90, 1)) + ).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(0L)); - searchResponse = client().prepareSearch("quad").setQuery( - geoShapeQuery("shape", new PointBuilder(-180, 1)) - ).get(); + searchResponse = client().prepareSearch("quad").setQuery( + geoShapeQuery("shape", new PointBuilder(-180, 1)) + ).get(); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); - searchResponse = client().prepareSearch("quad").setQuery( - geoShapeQuery("shape", new PointBuilder(180, 1)) - ).get(); + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); - assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + searchResponse = client().prepareSearch("quad").setQuery( + geoShapeQuery("shape", new PointBuilder(180, 1)) + ).get(); - searchResponse = client().prepareSearch("vector").setQuery( + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + } finally { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + } + + SearchResponse searchResponse = client().prepareSearch("vector").setQuery( geoShapeQuery("shape", new PointBuilder(90, 1)) ).get(); diff --git a/server/src/test/java/org/elasticsearch/search/geo/LegacyGeoShapeIntegrationIT.java b/server/src/test/java/org/elasticsearch/search/geo/LegacyGeoShapeIntegrationIT.java index 3aba9d6b259aa..a056d4a4227f9 100644 --- a/server/src/test/java/org/elasticsearch/search/geo/LegacyGeoShapeIntegrationIT.java +++ b/server/src/test/java/org/elasticsearch/search/geo/LegacyGeoShapeIntegrationIT.java @@ -18,11 +18,14 @@ */ package org.elasticsearch.search.geo; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.cluster.ClusterState; import org.elasticsearch.cluster.routing.IndexShardRoutingTable; import org.elasticsearch.common.Strings; import org.elasticsearch.common.geo.builders.ShapeBuilder; +import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.ToXContent; import org.elasticsearch.common.xcontent.XContentFactory; import org.elasticsearch.common.xcontent.XContentType; @@ -33,6 +36,8 @@ import org.elasticsearch.indices.IndicesService; import org.elasticsearch.test.ESIntegTestCase; +import java.io.IOException; + import static org.elasticsearch.index.query.QueryBuilders.geoShapeQuery; import static org.elasticsearch.index.query.QueryBuilders.matchAllQuery; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked; @@ -186,6 +191,53 @@ public void testLegacyCircle() throws Exception { assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); } + public void testDisallowExpensiveQueries() throws InterruptedException, IOException { + try { + // create index + assertAcked(client().admin().indices().prepareCreate("test") + .setMapping("shape", "type=geo_shape,strategy=recursive,tree=geohash").get()); + ensureGreen(); + + indexRandom(true, client().prepareIndex("test").setId("0").setSource( + "shape", (ToXContent) (builder, params) -> { + builder.startObject().field("type", "circle") + .startArray("coordinates").value(30).value(50).endArray() + .field("radius", "77km") + .endObject(); + return builder; + })); + refresh(); + + // Execute with search.allow_expensive_queries = null => default value = false => success + SearchResponse searchResponse = client().prepareSearch("test").setQuery(geoShapeQuery("shape", + new Circle(30, 50, 77000))).get(); + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", false)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + + // Set search.allow_expensive_queries to "false" => assert failure + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> client().prepareSearch("test").setQuery(geoShapeQuery("shape", + new Circle(30, 50, 77000))).get()); + assertEquals("[geo-shape] queries on [PrefixTree geo shapes] cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", e.getCause().getMessage()); + + // Set search.allow_expensive_queries to "true" => success + updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + searchResponse = client().prepareSearch("test").setQuery(geoShapeQuery("shape", + new Circle(30, 50, 77000))).get(); + assertThat(searchResponse.getHits().getTotalHits().value, equalTo(1L)); + } finally { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + } + } + private String findNodeName(String index) { ClusterState state = client().admin().cluster().prepareState().get().getState(); IndexShardRoutingTable shard = state.getRoutingTable().index(index).shard(0); diff --git a/server/src/test/java/org/elasticsearch/search/query/ScriptScoreQueryIT.java b/server/src/test/java/org/elasticsearch/search/query/ScriptScoreQueryIT.java index 48e8e9abfbb6b..65449430aa878 100644 --- a/server/src/test/java/org/elasticsearch/search/query/ScriptScoreQueryIT.java +++ b/server/src/test/java/org/elasticsearch/search/query/ScriptScoreQueryIT.java @@ -19,6 +19,8 @@ package org.elasticsearch.search.query; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.index.fielddata.ScriptDocValues; @@ -154,4 +156,55 @@ public void testRewrittenQuery() { assertNoFailures(resp); assertOrderedSearchHits(resp, "3", "2", "1"); } + + public void testDisallowExpensiveQueries() { + try { + assertAcked( + prepareCreate("test-index").setMapping("field1", "type=text", "field2", "type=double") + ); + int docCount = 10; + for (int i = 1; i <= docCount; i++) { + client().prepareIndex("test-index").setId("" + i) + .setSource("field1", "text" + (i % 2), "field2", i) + .get(); + } + refresh(); + + // Execute with search.allow_expensive_queries = null => default value = true => success + Script script = new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['field2'].value * param1", + Map.of("param1", 0.1)); + SearchResponse resp = client() + .prepareSearch("test-index") + .setQuery(scriptScoreQuery(matchQuery("field1", "text0"), script)) + .get(); + assertNoFailures(resp); + + // Set search.allow_expensive_queries to "false" => assert failure + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", false)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> client() + .prepareSearch("test-index") + .setQuery(scriptScoreQuery(matchQuery("field1", "text0"), script)) + .get()); + assertEquals("[script score] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getCause().getMessage()); + + // Set search.allow_expensive_queries to "true" => success + updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + resp = client() + .prepareSearch("test-index") + .setQuery(scriptScoreQuery(matchQuery("field1", "text0"), script)) + .get(); + assertNoFailures(resp); + } finally { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + } + } } diff --git a/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java b/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java index e22daab2e89cf..341202d44a940 100644 --- a/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java @@ -144,7 +144,7 @@ public void testBuildRescoreSearchContext() throws ElasticsearchParseException, // shard context will only need indicesQueriesRegistry for building Query objects nested in query rescorer QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null) { + xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); @@ -188,7 +188,7 @@ public void testRewritingKeepsSettings() throws IOException { // shard context will only need indicesQueriesRegistry for building Query objects nested in query rescorer QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null) { + xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); diff --git a/server/src/test/java/org/elasticsearch/search/scriptfilter/ScriptQuerySearchIT.java b/server/src/test/java/org/elasticsearch/search/scriptfilter/ScriptQuerySearchIT.java index 7456706a8b81d..baeed397204d5 100644 --- a/server/src/test/java/org/elasticsearch/search/scriptfilter/ScriptQuerySearchIT.java +++ b/server/src/test/java/org/elasticsearch/search/scriptfilter/ScriptQuerySearchIT.java @@ -19,6 +19,8 @@ package org.elasticsearch.search.scriptfilter; +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; @@ -47,6 +49,7 @@ import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder; import static org.elasticsearch.index.query.QueryBuilders.scriptQuery; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked; +import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertNoFailures; import static org.hamcrest.Matchers.equalTo; @ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.SUITE) @@ -222,6 +225,55 @@ public void testCustomScriptBoost() throws Exception { assertThat(response.getHits().getAt(2).getFields().get("sNum1").getValues().get(0), equalTo(3.0)); } + public void testDisallowExpensiveQueries() { + try { + assertAcked( + prepareCreate("test-index").setMapping("num1", "type=double") + ); + int docCount = 10; + for (int i = 1; i <= docCount; i++) { + client().prepareIndex("test-index").setId("" + i) + .setSource("num1", i) + .get(); + } + refresh(); + + // Execute with search.allow_expensive_queries = null => default value = false => success + Script script = new Script(ScriptType.INLINE, CustomScriptPlugin.NAME, "doc['num1'].value > 1", + Collections.emptyMap()); + SearchResponse resp = client().prepareSearch("test-index") + .setQuery(scriptQuery(script)) + .get(); + assertNoFailures(resp); + + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", false)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + + // Set search.allow_expensive_queries to "false" => assert failure + ElasticsearchException e = expectThrows(ElasticsearchException.class, + () -> client() + .prepareSearch("test-index") + .setQuery(scriptQuery(script)) + .get()); + assertEquals("[script] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + e.getCause().getMessage()); + + // Set search.allow_expensive_queries to "true" => success + updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", true)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + resp = client().prepareSearch("test-index") + .setQuery(scriptQuery(script)) + .get(); + assertNoFailures(resp); + } finally { + ClusterUpdateSettingsRequest updateSettingsRequest = new ClusterUpdateSettingsRequest(); + updateSettingsRequest.persistentSettings(Settings.builder().put("search.allow_expensive_queries", (String) null)); + assertAcked(client().admin().cluster().updateSettings(updateSettingsRequest).actionGet()); + } + } + private static AtomicInteger scriptCounter = new AtomicInteger(0); public static int incrementScriptCounter() { diff --git a/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java b/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java index 9cda381df243b..2ec4038529972 100644 --- a/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java +++ b/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java @@ -198,7 +198,7 @@ protected final QueryShardContext createMockShardContext(IndexSearcher searcher) }; return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, bitsetFilterCache, indexFieldDataLookup, null, null, scriptService, xContentRegistry(), namedWriteableRegistry, null, searcher, - () -> randomNonNegativeLong(), null, null) { + () -> randomNonNegativeLong(), null, null, () -> true) { @Override public MappedFieldType fieldMapper(String name) { diff --git a/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java b/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java index f373188de9fc4..9bbd291b32a0d 100644 --- a/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java +++ b/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java @@ -181,7 +181,7 @@ public void testBuild() throws IOException { ((Script) invocation.getArguments()[0]).getIdOrCode())); QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, scriptService, xContentRegistry(), namedWriteableRegistry, null, null, - System::currentTimeMillis, null, null); + System::currentTimeMillis, null, null, () -> true); SuggestionContext suggestionContext = suggestionBuilder.build(mockShardContext); assertEquals(toBytesRef(suggestionBuilder.text()), suggestionContext.getText()); diff --git a/test/framework/src/main/java/org/elasticsearch/index/mapper/FieldTypeTestCase.java b/test/framework/src/main/java/org/elasticsearch/index/mapper/FieldTypeTestCase.java index 8914bad5c4102..1b8f78c761f2e 100644 --- a/test/framework/src/main/java/org/elasticsearch/index/mapper/FieldTypeTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/index/mapper/FieldTypeTestCase.java @@ -31,9 +31,15 @@ import java.util.Arrays; import java.util.List; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + /** Base test case for subclasses of MappedFieldType */ public abstract class FieldTypeTestCase extends ESTestCase { + public static final QueryShardContext MOCK_QSC = createMockQueryShardContext(true); + public static final QueryShardContext MOCK_QSC_DISALLOW_EXPENSIVE = createMockQueryShardContext(false); + /** Abstraction for mutating a property of a MappedFieldType */ public abstract static class Modifier { /** The name of the property that is being modified. Used in test failure messages. */ @@ -243,6 +249,16 @@ protected String toString(MappedFieldType ft) { "} " + super.toString(); } + protected QueryShardContext randomMockShardContext() { + return randomFrom(MOCK_QSC, MOCK_QSC_DISALLOW_EXPENSIVE); + } + + static QueryShardContext createMockQueryShardContext(boolean allowExpensiveQueries) { + QueryShardContext queryShardContext = mock(QueryShardContext.class); + when(queryShardContext.allowExpensiveQueries()).thenReturn(allowExpensiveQueries); + return queryShardContext; + } + public void testClone() { MappedFieldType fieldType = createNamedDefaultFieldType(); MappedFieldType clone = fieldType.clone(); diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java index e77f406e82a28..3bffb7f7287bf 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java @@ -282,7 +282,7 @@ protected QueryShardContext queryShardContextMock(IndexSearcher searcher, return new QueryShardContext(0, indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, getIndexFieldDataLookup(mapperService, circuitBreakerService), mapperService, null, getMockScriptService(), xContentRegistry(), - writableRegistry(), null, searcher, System::currentTimeMillis, null, null); + writableRegistry(), null, searcher, System::currentTimeMillis, null, null, () -> true); } /** diff --git a/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java b/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java index a682cf4ea490d..875bb775ba21f 100644 --- a/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java @@ -413,7 +413,7 @@ public void close() throws IOException { QueryShardContext createShardContext(IndexSearcher searcher) { return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, bitsetFilterCache, indexFieldDataService::getForField, mapperService, similarityService, scriptService, xContentRegistry, - namedWriteableRegistry, this.client, searcher, () -> nowInMillis, null, indexNameMatcher()); + namedWriteableRegistry, this.client, searcher, () -> nowInMillis, null, indexNameMatcher(), () -> true); } ScriptModule createScriptModule(List scriptPlugins) { diff --git a/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java b/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java index 8a8842487f14a..684210f13c57f 100644 --- a/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java +++ b/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java @@ -43,7 +43,7 @@ public void testAssertNoInFlightContext() { final long nowInMillis = randomNonNegativeLong(); SearchContext s = new TestSearchContext(new QueryShardContext(0, new IndexSettings(EMPTY_INDEX_METADATA, Settings.EMPTY), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null)) { + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true)) { @Override public SearchShardTarget shardTarget() { diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java index 36d81e6ea9541..3cfab7ddcfe9e 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java @@ -24,8 +24,8 @@ import org.apache.lucene.util.BitSet; import org.elasticsearch.client.Client; import org.elasticsearch.common.CheckedBiConsumer; -import org.elasticsearch.common.logging.Loggers; import org.elasticsearch.common.CheckedConsumer; +import org.elasticsearch.common.logging.Loggers; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.util.BigArrays; import org.elasticsearch.index.IndexSettings; @@ -521,7 +521,7 @@ private TestIndexContext testIndex(MapperService mapperService, Client client) t final QueryShardContext shardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, new IndexSearcher(directoryReader), () -> nowInMillis, null, null); + client, new IndexSearcher(directoryReader), () -> nowInMillis, null, null, () -> true); context = new TestIndexContext(directory, iw, directoryReader, shardContext, leaf); return context; diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java index 36102090efed6..117fa4b2f49c9 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java @@ -84,7 +84,7 @@ public void testDLS() throws Exception { final long nowInMillis = randomNonNegativeLong(); QueryShardContext realQueryShardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, null, () -> nowInMillis, null, null); + client, null, () -> nowInMillis, null, null, () -> true); QueryShardContext queryShardContext = spy(realQueryShardContext); DocumentSubsetBitsetCache bitsetCache = new DocumentSubsetBitsetCache(Settings.EMPTY, Executors.newSingleThreadExecutor()); XPackLicenseState licenseState = mock(XPackLicenseState.class); @@ -209,7 +209,7 @@ public void testDLSWithLimitedPermissions() throws Exception { final long nowInMillis = randomNonNegativeLong(); QueryShardContext realQueryShardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, null, () -> nowInMillis, null, null); + client, null, () -> nowInMillis, null, null, () -> true); QueryShardContext queryShardContext = spy(realQueryShardContext); DocumentSubsetBitsetCache bitsetCache = new DocumentSubsetBitsetCache(Settings.EMPTY, Executors.newSingleThreadExecutor()); diff --git a/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java b/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java index 5928d9889c8c8..5c182284e0c20 100644 --- a/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java +++ b/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java @@ -306,7 +306,7 @@ public Query rangeQuery(Object lowerTerm, @Override public Query fuzzyQuery(Object value, Fuzziness fuzziness, int prefixLength, int maxExpansions, - boolean transpositions) { + boolean transpositions, QueryShardContext context) { throw new UnsupportedOperationException("[fuzzy] queries are not currently supported on keyed " + "[" + CONTENT_TYPE + "] fields."); } diff --git a/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/KeyedFlatObjectFieldTypeTests.java b/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/KeyedFlatObjectFieldTypeTests.java index 46901035c8a96..2a106d6a96c5f 100644 --- a/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/KeyedFlatObjectFieldTypeTests.java +++ b/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/KeyedFlatObjectFieldTypeTests.java @@ -15,6 +15,7 @@ import org.apache.lucene.search.TermQuery; import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.unit.Fuzziness; import org.elasticsearch.index.mapper.FieldTypeTestCase; import org.elasticsearch.index.mapper.MappedFieldType; @@ -98,7 +99,12 @@ public void testPrefixQuery() { ft.setName("field"); Query expected = new PrefixQuery(new Term("field", "key\0val")); - assertEquals(expected, ft.prefixQuery("val", MultiTermQuery.CONSTANT_SCORE_REWRITE, null)); + assertEquals(expected, ft.prefixQuery("val", MultiTermQuery.CONSTANT_SCORE_REWRITE, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.prefixQuery("val", MultiTermQuery.CONSTANT_SCORE_REWRITE, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[prefix] queries cannot be executed when 'search.allow_expensive_queries' is set to false. " + + "For optimised prefix queries on text fields please enable [index_prefixes].", ee.getMessage()); } public void testFuzzyQuery() { @@ -106,7 +112,7 @@ public void testFuzzyQuery() { ft.setName("field"); UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, - () -> ft.fuzzyQuery("valuee", Fuzziness.fromEdits(2), 1, 50, true)); + () -> ft.fuzzyQuery("value", Fuzziness.fromEdits(2), 1, 50, true, randomMockShardContext())); assertEquals("[fuzzy] queries are not currently supported on keyed [flattened] fields.", e.getMessage()); } @@ -117,12 +123,12 @@ public void testRangeQuery() { TermRangeQuery expected = new TermRangeQuery("field", new BytesRef("key\0lower"), new BytesRef("key\0upper"), false, false); - assertEquals(expected, ft.rangeQuery("lower", "upper", false, false, null)); + assertEquals(expected, ft.rangeQuery("lower", "upper", false, false, MOCK_QSC)); expected = new TermRangeQuery("field", new BytesRef("key\0lower"), new BytesRef("key\0upper"), true, true); - assertEquals(expected, ft.rangeQuery("lower", "upper", true, true, null)); + assertEquals(expected, ft.rangeQuery("lower", "upper", true, true, MOCK_QSC)); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> ft.rangeQuery("lower", null, false, false, null)); @@ -130,9 +136,14 @@ public void testRangeQuery() { e.getMessage()); e = expectThrows(IllegalArgumentException.class, () -> - ft.rangeQuery(null, "upper", false, false, null)); + ft.rangeQuery(null, "upper", false, false, MOCK_QSC)); assertEquals("[range] queries on keyed [flattened] fields must include both an upper and a lower bound.", e.getMessage()); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.rangeQuery("lower", "upper", false, false, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[range] queries on [text] or [keyword] fields cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", ee.getMessage()); } public void testRegexpQuery() { @@ -140,7 +151,7 @@ public void testRegexpQuery() { ft.setName("field"); UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, - () -> ft.regexpQuery("valu*", 0, 10, null, null)); + () -> ft.regexpQuery("valu*", 0, 10, null, randomMockShardContext())); assertEquals("[regexp] queries are not currently supported on keyed [flattened] fields.", e.getMessage()); } @@ -149,7 +160,7 @@ public void testWildcardQuery() { ft.setName("field"); UnsupportedOperationException e = expectThrows(UnsupportedOperationException.class, - () -> ft.wildcardQuery("valu*", null, null)); + () -> ft.wildcardQuery("valu*", null, randomMockShardContext())); assertEquals("[wildcard] queries are not currently supported on keyed [flattened] fields.", e.getMessage()); } } diff --git a/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/RootFlatObjectFieldTypeTests.java b/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/RootFlatObjectFieldTypeTests.java index be297663c6e74..e0afaf007a5f9 100644 --- a/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/RootFlatObjectFieldTypeTests.java +++ b/x-pack/plugin/mapper-flattened/src/test/java/org/elasticsearch/xpack/flattened/mapper/RootFlatObjectFieldTypeTests.java @@ -16,6 +16,7 @@ import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.search.WildcardQuery; import org.apache.lucene.util.BytesRef; +import org.elasticsearch.ElasticsearchException; import org.elasticsearch.common.unit.Fuzziness; import org.elasticsearch.index.mapper.FieldNamesFieldMapper; import org.elasticsearch.index.mapper.FieldTypeTestCase; @@ -78,8 +79,14 @@ public void testFuzzyQuery() { ft.setName("field"); Query expected = new FuzzyQuery(new Term("field", "value"), 2, 1, 50, true); - Query actual = ft.fuzzyQuery("value", Fuzziness.fromEdits(2), 1, 50, true); + Query actual = ft.fuzzyQuery("value", Fuzziness.fromEdits(2), 1, 50, true, MOCK_QSC); assertEquals(expected, actual); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.fuzzyQuery("value", Fuzziness.AUTO, randomInt(10) + 1, randomInt(10) + 1, + randomBoolean(), MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[fuzzy] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testRangeQuery() { @@ -89,12 +96,17 @@ public void testRangeQuery() { TermRangeQuery expected = new TermRangeQuery("field", new BytesRef("lower"), new BytesRef("upper"), false, false); - assertEquals(expected, ft.rangeQuery("lower", "upper", false, false, null)); + assertEquals(expected, ft.rangeQuery("lower", "upper", false, false, MOCK_QSC)); expected = new TermRangeQuery("field", new BytesRef("lower"), new BytesRef("upper"), true, true); - assertEquals(expected, ft.rangeQuery("lower", "upper", true, true, null)); + assertEquals(expected, ft.rangeQuery("lower", "upper", true, true, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.rangeQuery("lower", "upper", true, true, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[range] queries on [text] or [keyword] fields cannot be executed when " + + "'search.allow_expensive_queries' is set to false.", ee.getMessage()); } public void testRegexpQuery() { @@ -102,8 +114,13 @@ public void testRegexpQuery() { ft.setName("field"); Query expected = new RegexpQuery(new Term("field", "val.*")); - Query actual = ft.regexpQuery("val.*", 0, 10, null, null); + Query actual = ft.regexpQuery("val.*", 0, 10, null, MOCK_QSC); assertEquals(expected, actual); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.regexpQuery("val.*", randomInt(10), randomInt(10) + 1, null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[regexp] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } public void testWildcardQuery() { @@ -111,6 +128,11 @@ public void testWildcardQuery() { ft.setName("field"); Query expected = new WildcardQuery(new Term("field", new BytesRef("valu*"))); - assertEquals(expected, ft.wildcardQuery("valu*", null, null)); + assertEquals(expected, ft.wildcardQuery("valu*", null, MOCK_QSC)); + + ElasticsearchException ee = expectThrows(ElasticsearchException.class, + () -> ft.wildcardQuery("valu*", null, MOCK_QSC_DISALLOW_EXPENSIVE)); + assertEquals("[wildcard] queries cannot be executed when 'search.allow_expensive_queries' is set to false.", + ee.getMessage()); } } diff --git a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java index 6de6b66ab7156..baabbfd8d3e94 100644 --- a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java +++ b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java @@ -91,7 +91,7 @@ private void setup() { settings = createIndexSettings(); queryShardContext = new QueryShardContext(0, settings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - null, null, null, null, () -> 0L, null, null); + null, null, null, null, () -> 0L, null, null, () -> true); } public void testSimpleDateHisto() throws Exception { diff --git a/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/WatcherPluginTests.java b/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/WatcherPluginTests.java index 5575915188351..b54f3c78c3e6a 100644 --- a/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/WatcherPluginTests.java +++ b/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/WatcherPluginTests.java @@ -74,7 +74,8 @@ public void testWatcherDisabledTests() throws Exception { IndexSettings indexSettings = IndexSettingsModule.newIndexSettings(Watch.INDEX, settings); AnalysisRegistry registry = new AnalysisRegistry(TestEnvironment.newEnvironment(settings), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap(), emptyMap()); - IndexModule indexModule = new IndexModule(indexSettings, registry, new InternalEngineFactory(), Collections.emptyMap()); + IndexModule indexModule = new IndexModule(indexSettings, registry, new InternalEngineFactory(), Collections.emptyMap(), + () -> true); // this will trip an assertion if the watcher indexing operation listener is null (which it is) but we try to add it watcher.onIndexModule(indexModule);