Merge master & resolve conflicts #54002
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…#53519) * [DOCS] Added missing defintions and edited for consistency with the Kibana glossary. * Apply suggestions from code review Co-Authored-By: James Rodewig <[email protected]> * Added API key
This commit upgrades our Jackson dependency to 2.10.3 and our GeoIP2 dependency to 2.13.1.
This commit removes some code that is duplicated in the parsing of allocation commands in the cluster reroute API.
This reverts commit 7bc75f4 Signed-off-by: Mark Vieira <[email protected]>
Today the NodeConnectionsService emits a DEBUG-level log message each time it calls TransportService#connectToNode, which happens for every node in the cluster every ten seconds, and also at every cluster state update. That's a lot of log messages. Most of these calls are no-ops and can be ignored, but if the call was not a no-op then it may be worth investigating further. Since the logs do not distinguish the interesting and uninteresting cases, they are not useful. This commit distinguishes the two cases and pushes the noisy logging for the common no-op case down to TRACE level, leaving only useful and actionable information in the DEBUG-level logs.
Indexes values using size 3 ngrams and also stores the full original as a binary doc value. Wildcard queries operate by using a cheap approximation query on the ngram field followed up by a more expensive verification query using an automaton on the binary doc values. Also supports aggregations and sorting.
Keyword field values with length more than ignore_above are not indexed. But highlighters still were retrieving these values from _source and were trying to highlight them. This sometimes lead to errors if a field length exceeded max_analyzed_offset. But also this is a wrong behaviour to attempt to highlight something that was not ignored during indexing. This PR checks if a keyword value was ignored because of its length, and if yes, skips highlighting it. Closes elastic#43800
Add missing asScript() implementation for LIKE/RLIKE expressions. When LIKE/RLIKE are used for example in GROUP BY or are wrapped with scalar functions in a WHERE clause, the translation must produce a painless script which will be executed to implement the correct behaviour and previously this was completely missing, and as a consquence wrong results were silently (no error) returned. Fixes: elastic#53486
* [ML] Extend classification to support multiple classes Prepares classification analysis to support more than just two classes. It introduces a new parameter to the process config which dictates the `num_classes` to the process. It also changes the max classes limit to `30` provisionally. * We can't test cardinality is too high in the YML tests anymore * Extract max number of classes in a constant
Fix NPE when `null` is passed as a parameter for a parameterized pattern of LIKE/RLIKE. e.g.: `field LIKE ?` params=[null]` Check for null pattern in LIKE/RLIKE as for RLIKE (RegexpQuery) we get an IllegalArgumentExpression from Lucence but for LIKE (WildcardQuery) we get an NPE. Fixes: elastic#53557
Makes the following changes to the `remove_duplicates` token filter docs: * Rewrites description and adds Lucene link * Adds detailed analyze example * Adds custom analyzer example
Prior to this commit Watcher explicitly copied test between two projects with a copy task. This commit removes the explicit copy in favor of adding the Watcher tests to the available restResources that may be copied between projects. This is how inter-project dependencies should be modeled. However, only Watcher is included here since it is (currently) the only project with inter-project test dependencies.
This reverts commit 887541e.
The watcher TextTemplateEngine uses a fast path mechanism where it
checks for the existence of `{{` to decide if a mustache script
required compilation. This does not work for stored script, as the field
that is checked contains the id of the script, which means, the name of
the script is returned as its value.
This commit checks for the script type and does not involve this fast
path check if a stored script is used.
Closes elastic#40212
* Removes experimental. * Replaces `"v"` (for value) with `"m"` (for metric). * Move the note about tiebreaking into the list of limitations of the sort. * Explain how you ask for `metrics`. * Clean up some wording. * Link to the docs from `top_metrics`. Closes elastic#51813
This commit adjusts the aliases used for the ILM and SLM history indices to be hidden aliases. Also tweaks the configuration of the `IndexTemplateRegistry`s used by these history system to only upgrade the template from the master node, as documents are indexed from the master node, so the template version should only be upgraded from the master node.
I broke sorting aggregations by `doc_count` in elastic#51271 by mixing up true and false. This flips that comparison and adds a few tests to double check that we don't so this again.
This begins to clean up how `PipelineAggregator`s and executed. Previously, we would create the `PipelineAggregator`s on the data nodes and embed them in the aggregation tree. When it came time to execute the pipeline aggregation we'd use the `PipelineAggregator`s that were on the first shard's results. This is inefficient because: 1. The data node needs to make the `PipelineAggregator` only to serialize it and then throw it away. 2. The coordinating node needs to deserialize all of the `PipelineAggregator`s even though it only needs one of them. 3. You end up with many `PipelineAggregator` instances when you only really *need* one per pipeline. 4. `PipelineAggregator` needs to implement serialization. This begins to undo these by building the `PipelineAggregator`s directly on the coordinating node and using those instead of the `PipelineAggregator`s in the aggregtion tree. In a follow up change we'll stop serializing the `PipelineAggregator`s to node versions that support this behavior. And, one day, we'll be able to remove `PipelineAggregator` from the aggregation result tree entirely. Importantly, this doesn't change how pipeline aggregations are declared or parsed or requested. They are still part of the `AggregationBuilder` tree because *that* makes sense.
Also adds a few small notes to the documentation regarding potentially unintuitive behavior
PR elastic#53617 has yet to be backported so we should skip its integration test on nodes earlier than 8.0.0
The `force` value for the `version_type` parameter was deprecated in 6.8. This removes the value from the parameter definition.
This commit adds the configured server_name to the proxy mode info so that it can be exposed in the remote info API.
This commit updates the serialization version for the server name on the proxy mode info, used in the remote info API.
Role names are now compiled from role templates before role mapping is saved. This serves as validation for role templates to prevent malformed and invalid scripts to be persisted, which could later break authentication. Resolves: elastic#48773
This change re-enables bwc tests now that elastic#54007 is merged in 7x.
disable BWC in preparation for elastic#54027
Until elastic#53998 is backported to 7.x.
The snapshot stats response list of snapshot statuses is not ordered according to the given list of snapshot names so randomly we could mix up snapshot1 and snapshot2 when asserting on the stats. Fixed by getting each snapshot's stats individually. Closes elastic#54034
* Add support for normalisation to wildcard field * Tidied imports * Added docs about params * Fix outdated error message * Avoid normaliser butchering wildcard query special characters * Fix broken test expectations * Fix wrong toString method * Address review comments - common method for normalising wildcard patterns and checkCompatibility * Remove unused import
finish and reenable BWC after backport of elastic#54027
It is possible for ML jobs to open lazily if the "allow_lazy_open" option in the job config is set to true. Such jobs wait in the "opening" state until a node has sufficient capacity to run them. This commit fixes the bug that prevented datafeeds for jobs lazily waiting assignment from being started. The state of such datafeeds is "starting", and they can be stopped by the stop datafeed API while in this state with or without force. Fixes elastic#53763
When we index data frame analytics stats docs we do not need to refresh immediately.
This commit adds an explicit cancellation of the search task if the initial async search submit task is cancelled (connection closed by the user). This was previously done through the cancellation of the parent task but we don't handle grand-children cancellation yet so we have to manually cancel the search task in order to ensure that shard actions are cancelled too. This change can be considered as a workaround until elastic#50990 is fixed.
* Add validation for component templates This change adds validation to make sure that settings and mappings are correct in component template. It's done the same way as in index templates - code is reused. Reletes to elastic#53101 * Fix checkstyle violation * Update server/src/main/java/org/elasticsearch/cluster/metadata/MetaDataIndexTemplateService.java Co-Authored-By: Lee Hinman <[email protected]> * Update server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataIndexTemplateServiceTests.java Co-Authored-By: Lee Hinman <[email protected]> * Update server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataIndexTemplateServiceTests.java Co-Authored-By: Lee Hinman <[email protected]> * Update server/src/test/java/org/elasticsearch/cluster/metadata/MetaDataIndexTemplateServiceTests.java Co-Authored-By: Lee Hinman <[email protected]> * Update server/src/main/java/org/elasticsearch/cluster/metadata/MetaDataIndexTemplateService.java Co-Authored-By: Lee Hinman <[email protected]> Co-authored-by: Lee Hinman <[email protected]>
IndexTemplateMetaData x-content serialization is a bit confused and trappy at the moment, with multiple static methods on the Builder object which may or may not emit the template name and/or mapping types. This commit consolidates everything into a single toXContent() method on the base class, with different options passed through as ToXContent.Params. Deserialization is converted to use an ObjectParser, taking the template name as a context.
* add flush always output option that will flush the output printer after each debug message when enabled (disabled by default) * at debug output initializationtime, log debug output information about OS, JVM and default JVM timezone
…acy one (.ml-state) (elastic#54070)
DocsClientYamlTestSuiteIT sometimes fails for CCR related tests because tests are started before the license is fully applied and active within the cluster. The first tests to be executed then fails with the error noticed in elastic#53430. This can be easily reproduced locally by only running CCR docs tests. This commit adds some @before logic in DocsClientYamlTestSuiteIT so that it waits for the license to be active before running CCR tests. Closes elastic#53430
This change merges the "feature-internal-idp" branch into Elasticsearch. This introduces a small identity-provider plugin as a child of the x-pack module. This allows ES to act as a SAML IdP, for users who are authenticated against the Elasticsearch cluster. This feature is intended for internal use within Elastic Cloud environments and is not supported for any other use case. It falls under an enterprise license tier. The IdP is disabled by default. Co-authored-by: Ioannis Kakavas <[email protected]>
Upgrades our lucene dependency to the released 8.5.0 version.
Replaces a link to Logstash OSS-only content with a link to the general Logstash monitoring topic.
Conflicts: server/src/main/java/org/elasticsearch/search/SearchModule.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/cumulativecardinality/CumulativeCardinalityAggregatorTests.java x-pack/plugin/wildcard/src/main/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapper.java
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
62 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.