Remove types from index templates for x-pack features #38637
Description
The following x-pack features use templates with explicit mappings in the templates:
- Logstash Central Configuration (
[types removal] Logstash central management typeless API kibana#30546) (Central management typeless API logstash#10421) (Update logstash-management.json to use typeless template #38653) - logstash-management.json
- Monitoring (
Remove types from internal monitoring templates and bump to api 7 #39888) (blocked byIngest ingest then create index #39607,update to send api version 6 logstash#10518,[Monitoring] Adapt monitoring to read from .monitoring-*-7-* indices kibana#32514) - monitoring-alerts.json
- monitoring-beats.json
- monitoring-es.json
- monitoring-kibana.json
- monitoring-logstash.json
- Security (
Types removal security index template #39705) - security-index-template.json
- security_audit_log.json
- Watcher (
Remove the index type from internal watcher indexes #39761) - triggered-watches.json
- watch-history.json
- watches.json
We can not simply remove the type from mapping since if any of the consumers use the (deprecated) typed API's they will start to fail. We need to update the consumers to the typeless API's before we can remove the types from the templates. Failure to update the consumers to the typeless API's will result in deprecation warnings that consumers have no means to resolve.
The types in the templates are relatively harmless since they still work with the type present. These internally managed templates differ from user defined/external defined templates since those with mappings with types will be rejected. This is due to an implementation detail, and ideally the internally managed templates follow the same rules as user defined/external templates.
Each of these areas requires some domain knowledge of which parts of the Elastic stack is consuming/generating the documents for these templates.
For example:
Logstash Central Configuration - Kibana writes the documents and Logstash reads the documents so both of these need to be updated to the typeless API's before we can remove the types.