v5 Completion suggester error

[shavo007]

Elasticsearch version:
5.0.0-alpha5
Plugins installed: []

JVM version:
1.8
OS version:
Mac OS X El Capitan
Description of the problem including expected versus actual behavior:
https://www.elastic.co/guide/en/elasticsearch/reference/master/search-suggesters-completion.html

I was trying to uplift the below blog entry:

https://www.elastic.co/blog/you-complete-me

Steps to reproduce:
Simple hotels index with type hotels

Index creation

PUT /hotels
{
  "settings": {
    "analysis": {
      "analyzer": {
        "suggest_synonyms": {
          "type":      "custom",
          "tokenizer": "lowercase",
          "filter":    [ "my_synonyms" ]
        }
      },
      "filter": {
        "my_synonyms": {
          "type":     "synonym",
          "synonyms": [ "courtyard, marriot" ]
        }
      }
    }
  },
  "mappings": {
    "hotel" : {
      "properties" : {
        "name" : { "type" : "string" },
        "city" : { "type" : "string" },
        "name_suggest" : {
          "type" :            "completion",
          "analyzer" :  "suggest_synonyms",
          "search_analyzer" : "suggest_synonyms"
        }
      } 
    }
  }
}

Hotel doc

PUT /hotels/hotel/1
{
  "name" :         "Mercure Hotel Munich",
  "city" :         "Munich",
  "name_suggest" : { 
    "input" :      [ 
      "Mercure Hotel Munich", 
      "Mercure Munich" 
    ],
    "weight":      5
  }
}

Suggest query is

POST /hotels/_suggest?pretty
{
  "hotels" : {
    "prefix" : "coutr",
    "completion" : {
      "field" : "name_suggest",
        "payload": ["city"],
       "fuzzy" : {
                "fuzziness" : 2
            }
    }
  }
}

Error is:

       "reason": {
               "type": "illegal_argument_exception",
               "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [city] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
            }

Thanks,
Shane.

[clintongormley]

Hi @shavo007

Thanks for reporting. The reason for this exception was that the completion suggester was returning doc values as the payload. You're trying to return doc values for an analyzed string field, which doesn't support doc values and so has to uninvert the index into heap in order to be able to respond from fielddata instead.

Since alpha5 we have changed it to return the original document instead (#19536) and removed the payload option (#19877)

[eskibars]

Additionally, we'll need to mark the blog as deprecated so others don't trip up on this. I'll take care of that. Thanks @shavo007

[shavo007]

hi guys,

Thanks for the quick response.
Im a bit confused. I read the breaking changes and mentions payload:

https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_suggester.html#_completion_suggester_is_document_oriented

Is the doc wrong and I cannot use payload in the query?

#19536 example shows the src returned in the response. if i take out payload i get this response:

POST /hotels/_suggest?pretty
{
  "hotels" : {
    "prefix" : "hot",
    "completion" : {
      "field" : "name_suggest",
       "fuzzy" : {
                "fuzziness" : 2
            }
    }
  }
}

{
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hotels": [
      {
         "text": "hot",
         "offset": 0,
         "length": 3,
         "options": [
            {
               "text": "Hotel Monaco",
               "score": 10
            }
         ]
      }
   ]
}

[shavo007]

BTW, do I get a gift package as part of the pioneer program..... 😛

[areek]

Im a bit confused. I read the breaking changes and mentions payload:
https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_50_suggester.html#_completion_suggester_is_document_oriented
Is the doc wrong and I cannot use payload in the query?

The doc is wrong, thanks for pointing that out :) @shavo007, will fix this

[shavo007]

I tested the example here:

https://www.elastic.co/guide/en/elasticsearch/reference/master/search-suggesters-completion.html#querying

The response I get is different:

{
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "song-suggest": [
      {
         "text": "nir",
         "offset": 0,
         "length": 3,
         "options": [
            {
               "text": "Nirvana",
               "score": 34
            }
         ]
      }
   ]
}

[areek]

@shavo007 #19536 is not in v5.0-alpha5, it is in master, so it will be released in the next release of v5.0, hence you get different response

[shavo007]

no worries. looks like the docs are out of sync in a few places...