Support for Elasticsearch 6.3 "missing" in Composite Aggregation

[hanswieheiri]

NEST/Elasticsearch.Net version: 6.4.1

Elasticsearch version: 6.3.0

Description of the problem including expected versus actual behavior:
I am using ICompositeAggregationSource where the field missing_bucket was newly added (that's compatible with Elasticsearch version 6.4.0). In 6.3.0 that field was called missing.

Is there a way to force serialization that would serialize bool? MissingBucket { get; set; } as missing instead of missing_bucket.

Thanks for your help.

[russcam]

Relates elastic/elasticsearch#29465

missing and missing_bucket are not the same; the former allows to set an explicit value when the value is null, whilst the latter emits
an explicit null value for documents without a value for a field.

missing is deprecated and will be removed in 7.x. I'm not that keen on the idea of implementing an already deprecated feature. Thoughts @Mpdreamz, @codebrain ?

@hanswieheiri would sending a query with the low level client, and returning a high level response work in the meantime?

var searchResponse = client.LowLevel.Search<SearchResponse<MyDocument>>("index", "type",
    PostData.Serializable(new
    {
        aggs = new {
            my_buckets = new {
                composite = new {
                    sources = new[] {
                        new { 
                            product = new { 
                                terms = new { 
                                    field = "product",
                                    missing = "my_missing_value"
                                } 
                            } 
                        }
                    }
                }
            }
        }
    }));

which would serialize to

{
  "aggs": {
    "my_buckets": {
      "composite": {
        "sources": [
          {
            "product": {
              "terms": {
                "field": "product",
                "missing": "my_missing_value"
              }
            }
          }
        ]
      }
    }
  }
}

[hanswieheiri]

Hi @russcam I agree it's not quite the same. Using the low level client kind of makes sense but I have a complicated logic in my "query". I.e. I would have to rebuild a lot! Is it maybe possible to modify the JSON string before its sent to Elastic?

[Mpdreamz]

I'm OK with adding missing given the alternative is to drop down to the low level client as long as #1016 is also still an open feature request.

[hanswieheiri]

In the meantime I wrapped Elasticsearch.Net.SerializableData for modifying the json via string replace in the method Write(Stream writableStream, IConnectionConfigurationValues settings).

The wrapper I injected in DispatchAsync() (by inheriting from ElasticClient)

Ugly but it works for now.

[russcam]

We'll look to add this in for the 6.6.0 release @hanswieheiri

[russcam]

Closing this; will be in the next release, 6.6.0