[Stack Monitoring] Remove internal collectors #47214
Description
The work described in this issue is a breaking change. As such, it is intended to be released only in a major release (i.e. no backports).
We now have the ability to monitor Elasticsearch, Kibana, Logstash, Beats, and APM Server with Metricbeat. The previous method of monitoring stack products, using collection code internal to each product that shipped monitoring data to a custom Monitoring Bulk API endpoint, is now deprecated. And in the next major release (likely 8.0.0), all code related to this internal collection approach should be removed.
This issue tracks the removal of one specific item related to the internal collection approach: the various internal collectors implemented by the Monitoring plugin, to collect Elasticsearch monitoring metrics. Note that settings related to these collectors should also be removed from documentation.
By removing these collectors Elasticsearch will no longer be able to monitor itself. Instead, users will have to run Metricbeat along side Elasticsearch and enable the elasticsearch-xpack module. This module knows how to poll various Elasticsearch APIs to periodically collect monitoring metrics and ship them to the monitoring Elasticsearch cluster.