index.mapper.dynamic=false not respected in Elasticsearch 6.x

[bittusarkar]

Elasticsearch version (bin/elasticsearch --version): 6.5.3

Plugins installed: []

JVM version (java -version): 1.8.0_72

OS version (uname -a if on a Unix-like system): Windows 10

Description of the problem including expected versus actual behavior:
Setting the value of index setting "index.mapper.dynamic" to false does not work in Elasticsearch 6.x.

Steps to reproduce:
Execute the following commands:

PUT test_index
{
  "settings": {
    "index.mapper.dynamic": false
  }
}

PUT test_index/test_mapping/1
{
  "foo": "bar"
}

Elasticsearch 5.6.10 fails as expected with the following response:

{
  "error": {
    "root_cause": [
      {
        "type": "type_missing_exception",
        "reason": "type[test_mapping] missing",
        "index_uuid": "IMHwQyOvQiW25XRA-i7lSA",
        "index": "test_index"
      }
    ],
    "type": "type_missing_exception",
    "reason": "type[test_mapping] missing",
    "index_uuid": "IMHwQyOvQiW25XRA-i7lSA",
    "index": "test_index",
    "caused_by": {
      "type": "illegal_state_exception",
      "reason": "trying to auto create mapping, but dynamic mapping is disabled"
    }
  },
  "status": 404
}

Elasticsearch 6.5.3 succeeds (not expected) with the following response:

{
  "_index": "test_index",
  "_type": "test_mapping",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 0,
  "_primary_term": 1
}

[albertzaharovits]

@bittusarkar this setting has been removed in #25734 . See https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html#_schedule_for_removal_of_mapping_types for the plan. I think this explains the behavior you're seeing, the only type the index will have is named "test_mapping" . Trying to add another type will make ES complain:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Rejecting mapping update to [test_index] as the final mapping would have more than 1 type: [test_mapping, test_mapping2]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "Rejecting mapping update to [test_index] as the final mapping would have more than 1 type: [test_mapping, test_mapping2]"
  },
  "status" : 400
}

The setting is ignored because sadly there is no setting validation under the index.mapping namespace.

[bittusarkar]

Also, this setting is deprecated in Elasticsearch 6.x and removed in Elasticsearch 7.x. The deprecation message is Setting index.mapper.dynamic is deprecated since indices may not have more than one type anymore.. But that does not make sense as long as creation of index without a mapping is supported. I verified that creation of index without mapping is still supported in Elasticsearch 7.0.

[bittusarkar]

@albertzaharovits See my comment above. I would understand the setting is useless if the only configuration possible was an index having a single mapping. But that is not true. An index can also have zero mapping as I have shown above. As long as a zero mapping index configuration is allowed, having this setting makes sense.

[albertzaharovits]

@bittusarkar are you complaining that you can add the mapping to an index after it has been created?

Some might consider this a feature, but you're politely invited to open a new issue and state your case for how this behavior is causing unexpected behavior.

[bittusarkar]

I understand why the setting is removed. To make this deprecation air tight, I am proposing the index creation logic should fail if mapping is not provided. This setting disallows creation of mapping automatically. Now, say I create an index with no mapping. Then due to some bug, a mapping gets created automatically (this has happened to us in the past). Then when I actually try to create the correct mapping explicitly, it would fail saying I cannot add multiple mappings. If this setting works as it did in Elasticsearch 5.x, a mapping would not get created due to the bug and my explicit mapping creation would succeed. I hope I was able to explain the scenario. We won't hit this issue because we always create an index with the mapping. I reported this issue because I felt this was a bug in Elasticsearch.