Skip to content

Run aggs tests in aggs module #90851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 17, 2022
Merged

Run aggs tests in aggs module #90851

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
dd859b6
Run aggs tests in aggs module
nik9000 Oct 12, 2022
e59d77b
Fix doc
nik9000 Oct 12, 2022
51dabd4
Better grouping
nik9000 Oct 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion ...src/main/java/org/elasticsearch/gradle/internal/rest/compat/YamlRestCompatTestPlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ public class YamlRestCompatTestPlugin implements Plugin<Project> {
private static final Path RELATIVE_API_PATH = Path.of("rest-api-spec/api");
private static final Path RELATIVE_TEST_PATH = Path.of("rest-api-spec/test");
private static final Path RELATIVE_REST_API_RESOURCES = Path.of("rest-api-spec/src/main/resources");
private static final Path RELATIVE_REST_CORE = Path.of("rest-api-spec");
private static final Path RELATIVE_REST_XPACK_RESOURCES = Path.of("x-pack/plugin/src/test/resources");
private static final Path RELATIVE_REST_PROJECT_RESOURCES = Path.of("src/yamlRestTest/resources");
private static final int COMPATIBLE_VERSION = Version.fromString(VersionProperties.getVersions().get("elasticsearch")).getMajor() - 1;
Expand Down Expand Up @@ -142,7 +143,11 @@ public void apply(Project project) {
task.getOutputResourceDir().set(projectLayout.getBuildDirectory().dir(compatTestsDir.resolve("original").toString()));
task.setCoreConfigToFileTree(
config -> fileOperations.fileTree(
config.getSingleFile().toPath().resolve(RELATIVE_REST_API_RESOURCES).resolve(RELATIVE_TEST_PATH)
config.getSingleFile()
.toPath()
.resolve(RELATIVE_REST_CORE)
.resolve(RELATIVE_REST_PROJECT_RESOURCES)
.resolve(RELATIVE_TEST_PATH)
Copy link
Member Author

@nik9000 nik9000 Oct 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old path didn't exist. I could have merged the paths some more but I don't feel particularly confident working on this code. I am a little worried what'll happen when I "fix" this. Will we start running tests we hadn't been running before? CI will tell me if any of them fail at least....

Copy link
Contributor

@mark-vieira mark-vieira Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so this was just wrong before I assume?

Copy link
Member Author

@nik9000 nik9000 Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so! The path didn't exist and nothing was ever copied.

Copy link
Contributor

@jakelandis jakelandis Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed this is a bug. Fortunately it only impacts copying tests from core (includeCore) for modules that have the yaml-rest-compat-test plugin. It seems that this only impacts this aggs module.

However, while reviewing i see that we have 2 projects (enrich, watcher with security) attempting to copy some tests from xpack (includeXpack) that are not running the compat tests as they should be since the path to the x-pack tests is also wrong. I will followup on that in a different PR.

Copy link
Contributor

@jakelandis jakelandis Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#90925 to fix the x-pack path, and #90929 to be a bit smarter here.

)
);
task.setXpackConfigToFileTree(
Expand Down
19 changes: 19 additions & 0 deletions modules/aggregations/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,23 @@ restResources {
restApi {
include '_common', 'indices', 'cluster', 'index', 'search', 'nodes', 'bulk'
}
restTests {
// Pulls in all aggregation tests from core AND the forwards v7's core for forwards compatibility
includeCore 'search.aggregation'
}
}

tasks.named("yamlRestTestV7CompatTransform").configure { task ->
task.skipTest("search.aggregation/20_terms/string profiler via global ordinals filters implementation", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/string profiler via global ordinals native implementation", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/string profiler via map", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/numeric profiler", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/210_top_hits_nested_metric/top_hits aggregation with sequence numbers", "#42809 the use nested path and filter sort throws an exception")
task.skipTest("search.aggregation/370_doc_count_field/Test filters agg with doc_count", "Uses profiler for assertions which is not backwards compatible")

task.addAllowedWarningRegex("\\[types removal\\].*")
Copy link
Member Author

@nik9000 nik9000 Oct 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were all from the rest-api-spec module

}

artifacts {
restTests(new File(projectDir, "src/yamlRestTest/resources/rest-api-spec/test"))
}
11 changes: 1 addition & 10 deletions rest-api-spec/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,6 @@ artifacts {

testClusters.configureEach {
module ':modules:mapper-extras'
// This dependency can be removed when aggregations and
// respective yaml test have been moved to aggregations module.
// See for the progress: https://github.com/elastic/elasticsearch/issues/90283
module ':modules:aggregations'
Copy link
Member

@martijnvg martijnvg Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

requiresFeature 'es.index_mode_feature_flag_registered', Version.fromString("8.0.0")
}

Expand All @@ -47,6 +43,7 @@ tasks.named("yamlRestTestV7CompatTransform").configure { task ->
task.skipTestsByFilePattern("**/cat*/*.yml", "Cat API are meant to be consumed by humans, so will not be supported by Compatible REST API")
task.skipTestsByFilePattern("**/indices.upgrade/*.yml", "upgrade api will only get a dummy endpoint returning an exception suggesting to use _reindex")
task.skipTestsByFilePattern("**/indices.stats/60_field_usage/*/*.yml", "field usage results will be different between lucene versions")
task.skipTestsByFilePattern("**/search.aggregation/*.yml", "run by the aggregation module")

task.skipTest("bulk/11_dynamic_templates/Dynamic templates", "Error message has changed")
task.skipTest("indices.create/20_mix_typeless_typeful/Implicitly create a typed index while there is a typeless template", "Type information about the type is removed and not passed down. The logic to check for this is also removed.")
Expand All @@ -70,18 +67,12 @@ tasks.named("yamlRestTestV7CompatTransform").configure { task ->
task.skipTest("indices.stats/20_translog/Translog retention settings are deprecated", "translog settings removal is not supported under compatible api")
task.skipTest("indices.stats/20_translog/Translog retention without soft_deletes", "translog settings removal is not supported under compatible api")
task.skipTest("indices.stats/20_translog/Translog stats on closed indices without soft-deletes", "translog settings removal is not supported under compatible api")
task.skipTest("search.aggregation/370_doc_count_field/Test filters agg with doc_count", "Uses profiler for assertions which is not backwards compatible")
task.skipTest("indices.create/10_basic/Create index without soft deletes", "Make soft-deletes mandatory in 8.0 #51122 - settings changes are note supported in Rest Api compatibility")
task.skipTest("field_caps/30_filter/Field caps with index filter", "behaviour change after #63692 4digits dates are parsed as epoch and in quotes as year")
task.skipTest("indices.forcemerge/10_basic/Check deprecation warning when incompatible only_expunge_deletes and max_num_segments values are both set", "#44761 bug fix")
task.skipTest("search/340_type_query/type query", "#47207 type query throws exception in compatible mode")
task.skipTest("search.aggregation/210_top_hits_nested_metric/top_hits aggregation with sequence numbers", "#42809 the use nested path and filter sort throws an exception")
task.skipTest("search/310_match_bool_prefix/multi_match multiple fields with cutoff_frequency throws exception", "#42654 cutoff_frequency, common terms are not supported. Throwing an exception")
task.skipTest("search_shards/10_basic/Search shards aliases with and without filters", "Filter representation no longer outputs default boosts")
task.skipTest("search.aggregation/20_terms/string profiler via global ordinals filters implementation", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/string profiler via global ordinals native implementation", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/string profiler via map", "The profiler results aren't backwards compatible.")
task.skipTest("search.aggregation/20_terms/numeric profiler", "The profiler results aren't backwards compatible.")
task.skipTest("migration/10_get_feature_upgrade_status/Get feature upgrade status", "Awaits backport")
task.skipTest("search/330_fetch_fields/Test disable source", "Error no longer thrown")
task.skipTest("search/240_date_nanos/doc value fields are working as expected across date and date_nanos fields", "Fetching docvalues field multiple times is no longer allowed")
Expand Down
1 change: 1 addition & 0 deletions x-pack/rest-resources-zip/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ configurations {
dependencies {
apis project(path: ':rest-api-spec', configuration: 'restSpecs')
freeTests project(path: ':rest-api-spec', configuration: 'restTests')
freeTests project(path: ':modules:aggregations', configuration: 'restTests')
Copy link
Member Author

@nik9000 nik9000 Oct 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The clients were going to lose tests every time we port things from rest-api-spec into the aggregations module without this.

compatApis project(path: ':rest-api-spec', configuration: 'restCompatSpecs')
compatApis project(path: ':x-pack:plugin', configuration: 'restCompatSpecs')
freeCompatTests project(path: ':rest-api-spec', configuration: 'restCompatTests')
Expand Down