Option to prevent auto-creating index with no matching index template

[axw]

Description

APM uses bulk indexing to data streams, and relies on automatic data stream creation on first document write. Index templates are set up by a different process (Kibana) than the one that writes the documents (APM Server).

If the index template is not created before a document is written, then a plain old index will be auto-created with default dynamic mapping rules which do match the expectations of our queries. To address this, we provide monitor privileges to APM Server so that it can query for the presence of index templates before writing documents. This is not ideal, as it increases the privileges (and therefore attack surface), and also more tightly couples APM Server to the index templates than should be necessary.

We would like a way to prevent writes from auto-creating an index if there is no matching index template. This might mean failing the request, or blocking the request for a configurable amount of time until a matching template has been installed.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[axw]

Probably a simpler option for us would be to introduce an index template built into Elasticsearch that matches traces-*-*, like we have for logs and metrics. I still think it could be a useful feature, but we wouldn't need it for APM if we had a built-in template.

EDIT: we did that, turns out it's insufficient as there's still a time window where the templates/pipelines don't exist while data can still be indexed.