Task Management Experimental Status

[henningandersen]

This is an experimental status issue, documenting the known remains before the Task management API can move out of experimental/beta state.

The primary outstanding items on the API is cementing the intentions of the API usage and its backwards compatibility guarantees. This was discussed in some detail here. This in turn adds work in several areas.

Following are the expected remaining actions before removing the experimental/beta status:

• Reframe the tasks API as a powerful troubleshooting tool that should not be used for building applications (discuss + docs).
• Clarify in docs that only the basic structure of the tasks API follows our backwards compatibliity principles. The structure (parent/child) of tasks, action names, descriptions, status and more can change in any release, including potentially micro/bugfix.
• Remove node and id from response (Remove node and id from list task response #31253)
• Ensure we do not use ListTasksResponse as response type for rethrottle (part of Reindex resiliency #42612)
• Add explicit cancel APIs where appropriate to not point users to the task cancel API.
• Similarly add explicit APIs for obtaining the result of tasks submitted with wait_for_completion=false (at least update and delete-by-query) and a mechanism for cleaning up older results once no longer needed.
• Remove the ability to view (and clean up) persisted task results, since these should be done through dedicated APIs (notice: the ephemeral task id is inherently unsafe to use, since it may be reused after node restarts).
• (optional) Add manage_tasks security privilege.

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Task Management)