PUT mapping with with search_analyzer and `update_all_types` does not actually update all types

[brwe]

Or toXContent is broken.

Reproduce:

DELETE test_index

PUT test_index
{
  "mappings": {
    "type1": {
      "properties": {
        "field": {
          "type": "string"
        }
      }
    },
    "type2": {
      "properties": {
        "field": {
          "type": "string"
        }
      }
    }
  }
}

GET test_index/_mapping/field/field?include_defaults

PUT test_index/type1/_mapping?update_all_types
{
  "properties": {
    "field": {
      "type": "string",
      "search_analyzer": "keyword",
      "analyzer": "default"
    }
  }
}

In the mapping I retrieve with the GET _mapping API in the mapping only type1 is updated, although with the GET field mapping API I get the correct mapping.

GET test_index/_mapping

{
  "test_index": {
    "mappings": {
      "type1": {
        "properties": {
          "field": {
            "type": "string",
            "analyzer": "default",
            "search_analyzer": "keyword"
          }
        }
      },
      "type2": {
        "properties": {
          "field": {
            "type": "string"
          }
        }
      }
    }
  }
}

GET test_index/_mapping/field/field


{
  "test_index": {
    "mappings": {
      "type2": {
        "field": {
          "full_name": "field",
          "mapping": {
            "field": {
              "type": "string",
              "analyzer": "default",
              "search_analyzer": "keyword"
            }
          }
        }
      },
      "type1": {
        "field": {
          "full_name": "field",
          "mapping": {
            "field": {
              "type": "string",
              "analyzer": "default",
              "search_analyzer": "keyword"
            }
          }
        }
      }
    }
  }
}

When I restart the node the change has vanished from the field mapping result but still is in the get mapping result.

[brwe]

tested only 2.1.1

[rjernst]

I tried on 2.2 and see the same behavior. The get mapping api is implemented by looking at the cluster state from the master, and doing a manual inspection of the cluster state shows the same inconsistency.

I have a hunch this is a problem in MetaDataMappingService.PutMappingExecutor, where I believe it should now simply be regenerating the entire mappings for the given index instead of the current complicated merging into the existing MappingMetaData of the index based on the type in the request. /cc @jpountz

[jpountz]

Good catch @rjernst, this was indeed one of the causes of the problem. I opened #16264 for it.

But there was also a serialization bug as Britta initially thought, which would be fixed by #16255.