Disabling _field_names don't work?

[nfsec]

Elasticsearch version (bin/elasticsearch --version): 6.1.3

Plugins installed: [] X-Pack Basic License

JVM version (java -version): 1.8.0_92

OS version (uname -a if on a Unix-like system): Ubuntu 16.04 LTS

Description of the problem including expected versus actual behavior:

I try to tune my Elasticsearch for indexing speed so I have disabled _field_names in template:

"index_patterns": [ "logs*" ], "aliases": {}, "mappings": { "doc": { "_field_names": { "enabled": false }, "dynamic_templates": [ { "strings": { "mapping": { "type": "keyword" }, "match_mapping_type": "string" } } ],

I see settings about _field_names in index mapping:

{ "logs_www-2018.05.25": { "mappings": { "doc": { "_field_names": { "enabled": false }, "dynamic_templates": [ { "strings": { "mapping": { "type": "keyword" }, "match_mapping_type": "string" } } ], "properties": { "@timestamp": { "type": "date" }, "level": { "type": "keyword" }, "message": { "analyzer": "logster", "type": "text", "norms": false }, "tag": { "type": "keyword", "ignore_above": 64 } }, "_source": { "excludes": [ "debug" ] } } } } }

But I'm still able to run exists queries on that index:

{ "query": { "exists" : { "field" : "level" } } }
So I guess _field_names are still active and add some overhead to indexing speed.

Steps to reproduce:

1. Create template with disabled _field_names: https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping-field-names-field.html
2. Create index with proper pattern.
3. Run exists query on any existing field.

[elasticmachine]

Pinging @elastic/es-search-aggs

[colings86]

From 6.1.0 we changed the way exists queries work in Elasticsearch so we rarely rely on _field_names. See #26930 and #27237 for more details but the essence of the change is that if a field has doc_values enabled (or for text fields if norms are enabled), we use a DocValuesFieldExistsQuery (or NormsFieldExistsQuery) instead of relying on _field_names. In fact we now don't even write the field name to _field_names if DocValues or norms are enabled to save the indexing overhead. This means that you can no get the functionality of the exists query without the indexing overhead of _field_names in most cases.

If you perform the exists query on the message field instead it should fail because you have disabled norms on that field as well as disabling _field_names

[nfsec]

@colings86 Thank you very much for the explanation. Can we please add this to documentation? This is very important information, because it turns out that we do not necessarily giving up _exists_ queries after disabling _field_names as it is written in https://www.elastic.co/guide/en/elasticsearch/reference/6.1/mapping-field-names-field.html. And users are not likely to follow commits on elastic repo - but read documentation.

[colings86]

@nfsec I opened #31029 to correct the wording fo the docs, thanks for bringing up the suggestion.