-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Description
There are some code snippets that we currently cannot test because they require time to run and generate data.
For example, see #31439, which describes why testing could not be enabled for a forecasting scenario. The same is true for several machine learning APIs that try to query results (e.g. https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-get-job-stats.html).
@nik9000 noted that some core tests work around this with stuff like using the health api to wait for green. For example:
> - do
cluster.health:
wait_for_status: yellow
There do not seem to be similar wait options for the machine learning case, however, so this issue proposes that we perhaps add a setup section when code snippet testing begins (e.g. to create some indices, add some data, and launch a simple machine learning job) and then not clean up those resources until the end of all the code snippet tests. This should allow sufficient time for some results to be generated that we can subsequently query in various pages in the documentation. NOTE: This setup would need to be skipped if -Dtests.distribution=oss-zip per #31619