Multi-fields will create field names with dots '.'

[brusic]

One of the breaking changes of Elasticsearch 2.0 was that field names are no longer allowed to have dots: #12068

However, if you use the new multi-field syntax, Elasticsearch will create field name with dots. The documentation supports the behavior. The aggregation in the example is on a field named "city.raw":

https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html

The unit tests in the code look for field names with dots: https://github.com/elastic/elasticsearch/blob/2.1/core/src/test/java/org/elasticsearch/index/mapper/multifield/MultiFieldTests.java#L88

The tests pass because the check in ObjectMapper$TypeParser only checks the original name.

The field name is built in ContentPath:
https://github.com/elastic/elasticsearch/blob/2.1/core/src/main/java/org/elasticsearch/index/mapper/ContentPath.java#L50
https://github.com/elastic/elasticsearch/blob/2.1/core/src/main/java/org/elasticsearch/index/mapper/ContentPath.java#L80-L87

These field names will fail if the indices are run through the MetaDataIndexUpgradeService. Are dots allowed in this context?

[clintongormley]

well spotted, thanks! Simple recreation, that runs on 2.1 (or on 1.7 and upgrades without warning to 2.1):

PUT t
{
  "mappings": {
    "t": {
      "properties": {
        "foo": {
          "type": "string",
          "fields": {
            "bar.baz": {
              "type": "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

PUT t/t/1
{
  "foo": "test"
}

GET t/t/_search
{
  "query": {
    "match": {
      "foo.bar.baz": "test"
    }
  }
}

[dadoonet]

I think it's the same root cause as described for the mapper attachment plugin. Linking to it here elastic/elasticsearch-mapper-attachments#169

[brwe]

I took a look and must admit I do not fully understand what the issue is.

@brusic What exactly is the error you see? I wrote a test for the example and all seems well (brwe@43e9426#diff-523d9598b02bfef9341f7aa20ebfdf47R543). The ContentPath builds the field path not the name and if I understand correctly that contains as many dots as there are levels.
I think I am missing something?

@clintongormley We should not be able to create multi fields with dots in the name, that is a bug for sure. I'll try to fix

[brusic]

It all depends on how strict should Elasticsearch should be regarding non-dots-in-field names. Better yet, what is the difference between a field name and a path?

At the Lucene level, multifield will create a field name with a dot in it. Inspected a one-shard index with Luke, and the fields created using the fields syntax will have a dot. Clinton's example does not fully show the impact since it uses a field name with a dot. Using 'bar' instead of 'bar.baz'.

{
  "mappings": {
    "t": {
      "properties": {
        "foo": {
          "type": "string",
          "fields": {
            "bar": {
              "type": "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

PUT t/t/1
{
  "foo": "test"
}

GET t/t/_search
{
  "query": {
    "match": {
      "foo.bar": "test"
    }
  }
}

A field name of foo.bar was created. Is this field name allowed? There was a discussion on the mailing list were these field names were getting flagged as incorrect when a user attempted to upgrade:

https://discuss.elastic.co/t/elasticsearch-2-mapping/34366/9

Either dots should never be allowed or the fields created with multifields should not be flagged as incorrect since they are paths and not field names.