Removed setting `index.mapper.dynamic` can be set on v7+ indices

[matt-monacelli]

Elasticsearch Version

7.10.2

Installed Plugins

No response

Java Version

bundled

OS Version

AWS 7.10.2

Problem Description

After setting a deprecated setting index.mapper.dynamic to false, I am unable to remove the setting and it is preventing me from opening the index:

{ "error" : { "root_cause" : [ { "type" : "exception", "reason" : "Failed to verify index [events-000012/e57doKb8Qe2p-D-fOLUcNw]" } ], "type" : "exception", "reason" : "Failed to verify index [events-000012/e57doKb8Qe2p-D-fOLUcNw]", "caused_by" : { "type" : "illegal_argument_exception", "reason" : "Setting index.mapper.dynamic was removed after version 6.0.0" } }, "status" : 500 }

Steps to Reproduce

1. Set index setting index.mapper.dynamic to false
2. close the index
3. try to open the index

Logs (if relevant)

No response

Workaround

Restore the problematic index from snapshot, passing "ignore_index_settings": "index.mapper.dynamic" to the restore request to filter out the bad setting.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[thecoop]

@matt-monacelli This setting was removed in v6 by #25734, which versions are you upgrading from?

[matt-monacelli]

@thecoop I am not upgrading, I have been on v7.10.2 and did not realize this setting was deprecated when I executed the API command to set it - now I am unable to set it back to true (or to remove it altogether by setting to null).

[DaveCTurner]

Relates the discussion at https://discuss.elastic.co/t/unable-to-open-index-after-config-change-while-closed/333325. I'm not convinced this is a bug, or at least not a bug which exists in any non-EOL versions today. For example in 7.17.10 you cannot set this setting even when the index is closed:

PUT /testindex
{
  "settings": {
    "number_of_replicas": 0
  }
}

# 200 OK
# {
#   "shards_acknowledged": true,
#   "acknowledged": true,
#   "index": "testindex"
# }

POST /testindex/_close

# 200 OK
# {
#   "shards_acknowledged": true,
#   "acknowledged": true,
#   "indices": {
#     "testindex": {
#       "closed": true
#     }
#   }
# }

PUT /testindex/_settings
{
  "index.mapper.dynamic": false
}

# 400 Bad Request
# {
#   "status": 400,
#   "error": {
#     "reason": "Setting index.mapper.dynamic was removed after version 6.0.0",
#     "root_cause": [
#       {
#         "reason": "Setting index.mapper.dynamic was removed after version 6.0.0",
#         "type": "illegal_argument_exception"
#       }
#     ],
#     "type": "illegal_argument_exception"
#   }
# }

Since I don't see that there's any action for the ES dev team to take here, I'm closing this.

[DaveCTurner]

Oh wait, you can't set it if the index is closed, but bizarrely you can set it if the index is open?! Yes, that's definitely a bug.

PUT /testindex
{
  "settings": {
    "number_of_replicas": 0
  }
}

# 200 OK
# {
#   "shards_acknowledged": true,
#   "acknowledged": true,
#   "index": "testindex"
# }

PUT /testindex/_settings
{
  "index.mapper.dynamic": false
}

# 200 OK
# {
#   "acknowledged": true
# }

POST /testindex/_close

# 200 OK
# {
#   "shards_acknowledged": true,
#   "acknowledged": true,
#   "indices": {
#     "testindex": {
#       "closed": true
#     }
#   }
# }

POST /testindex/_open

# 500 Internal Server Error
# {
#   "status": 500,
#   "error": {
#     "caused_by": {
#       "reason": "Setting index.mapper.dynamic was removed after version 6.0.0",
#       "type": "illegal_argument_exception"
#     },
#     "reason": "Failed to parse mappings for index [[testindex/onSqFfMbSmKWfkUrel8UhQ]]",
#     "root_cause": [
#       {
#         "reason": "Failed to parse mappings for index [[testindex/onSqFfMbSmKWfkUrel8UhQ]]",
#         "type": "illegal_state_exception"
#       }
#     ],
#     "type": "illegal_state_exception"
#   }
# }