Missing rest specs for testing features' code snippets

[lcawl]

The docs/build.gradle that we use for code snippet testing doesn't seem to parse all the X-Pack API specs. For example, in #44018 we received:

org.elasticsearch.smoketest.DocsClientYamlTestSuiteIT > test {yaml=reference/ml/apis/delete-dfanalytics/line_38} FAILED
08:43:32 java.lang.RuntimeException: Failure at [reference/ml/apis/delete-dfanalytics:33]: rest api [ml.put_data_frame_analytics] doesn't exist in the rest spec

We have worked around this for rollups and for a handful of ML APIs by using "raw" methods in the build.gradle file. For example: #48259

Instead of implementing these work-arounds, however, it would be preferable to have all the rest specs recognized by the documentation tests.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Build)

[lcawl]

Per @droberts195:

We have cleanup code both for the X-Pack REST tests [1] and the (Apache 2 license) HLRC tests [2].

The docs tests in the x-pack/docs directory already use the X-Pack ML REST test cleaner, because XDocsClientYamlTestSuiteIT extends XPackRestIT [3] and that uses the MlRestTestStateCleaner.

The only ML cleanup for the main docs directory tests is to wait for pending tasks [4]. So ironically the x-pack/docs tests are doing ML cleanup after each test (don’t worry - it should be very fast when there are no ML jobs) whereas the tests of the directory where the ML documentation actually lives are not.

One way we could solve both the cleanup problem and the spec problem would be to move all the ML docs into the x-pack/docs directory...The other way would be to either duplicate the HLRC ML cleanup class into the docs test code or put it somewhere where both sets of tests can use it...

[1] https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ml/integration/MlRestTestStateCleaner.java#L19
[2] https://github.com/elastic/elasticsearch/blob/master/client/rest-high-level/src/test/java/org/elasticsearch/client/MlTestStateCleaner.java#L42
[3] https://github.com/elastic/elasticsearch/blob/master/x-pack/docs/src/test/java/org/elasticsearch/smoketest/XDocsClientYamlTestSuiteIT.java#L32
[4] https://github.com/elastic/elasticsearch/blob/master/docs/src/test/java/org/elasticsearch/smoketest/DocsClientYamlTestSuiteIT.java#L103

[elasticmachine]

Pinging @elastic/es-docs (:Docs)

[mark-vieira]

@lcawl Does this continue to be a problem? I think with the change to remove the OSS distribution this simplifies things a bit WRT to testing as we no longer will have the notion of "OSS" docs.

@jakelandis on a related point, I wonder to what degree we can simplify the distinction between "oss" and "x-pack" for REST API specs in generally, given that the default distribution is now the only one that will exist 🤔

[lcawl]

I haven't tried removing them lately, but I still see "raw" methods in the build.gradle, such as

elasticsearch/docs/build.gradle

Line 762 in 974c9aa

raw: