Template .monitoring-es conflicts with my template

[mars00772]

By using ES 5.5 with x-pack installed, I got so many error log telling timestamp mapper conflicts:

[2017-08-18T11:43:24,474][WARN ][o.e.x.m.e.l.LocalExporter] unexpected error while indexing monitoring document
org.elasticsearch.xpack.monitoring.exporter.ExportException: java.lang.IllegalArgumentException: Mapper for [timestamp] conflicts with existing mapping in other types:
[mapper [timestamp] is used by multiple types. Set update_all_types to true to update [format] across all types.]
        at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$throwExportException$2(LocalBulk.java:130) ~[x-pack-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk$$Lambda$2137/1860532805.apply(Unknown Source) ~[?:?]

Just a wild guess,I think the reason was template .monitoring-es conflicts with my template
x-pack generated a template like this for the monitor index

  ".monitoring-es": {
    "order": 0,
    "version": 5050099,
    "template": ".monitoring-es-6-*",
  "mappings": {
      "doc": {
        "properties": {
          "timestamp": {
            "format": "date_time",
            "type": "date"
          }
        }
      }
    }
....

I have a template like this, for my indices:

   "t1":{
   "template" : "*",
    "order": 0,
    "mappings": {
      "log": {
        "properties": {
          "timestamp": {
            "format": "epoch_millis",
            "type": "date"
          }
...

If what I suspect is correct, puting new high order template shoud work, which it didn't.
what shoud be done? @warkolm thanks for helping.

curl -XPUT http://xxxx:9400/_template/fix_monitor?master_timeout=5m '-d
{
  "template" : ".monitoring-es-6-*", // same template name,but high oder 
  "order": 1,
    "mappings": {
      "doc": {
        "properties": {
          "timestamp": {
            "format": "date_time",
            "type": "date"
          }
        }
      }
    }
}'

[tlrx]

Just a wild guess,I think the reason was template .monitoring-es conflicts with my template

That's a good guess but it's a bit different: the custom template is not restrictive enough and conflicts with the X-Pack Monitoring template.

X-Pack exports document in indices that starts with the prefix .monitoring-es-6-. So Elasticsearch looks up for template that can be applied to such indices and finds ".monitoring-es" and "t1". Then it merges the template to build the final mappings, and a conflict error occurs because the same field timestamp is defined for the same document type doc (yes, X-Pack uses doc as document type too).

The best solution, I think, would be to change the t1 template so that it is applied only to your indices.

[mars00772]

@tlrx why the high order templatefix_monitor didn't work for .monitoring-es, and bad news for me, there are too many indices, which didn't have the common ##patten

[tlrx]

It didn't work because fields with same name but different type must have the same settings (see #11812), in order to avoid all kind of inconsistencies when querying the field. In your case, the custom template is applied to the monitoring indices too (but it should not) and it causes the conflict.