[Rollup] An empty index (or one entirely missing the timestamp) causes search exception

[polyfractal]

If an index is either empty, or is entirely filled with documents that lack the timestamp field, the indexer throws search exceptions:

reason [RemoteTransportException[[qfVIsK3][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[failed to find field [utc_time] and [missing_bucket] is not set]; ], cause [[logstash-2015.05.21/ciS2eEMfRW-IAfeVA8Xd8g] QueryShardException[failed to find field [utc_time] and [missing_bucket] is not set]
at org.elasticsearch.search.aggregations.bucket.composite.CompositeValuesSourceBuilder.build(CompositeValuesSourceBuilder.java:336)

We set missing_bucket for the other grouping fields so that we can record missing/null buckets. But because the timestamp is mandatory we don't set missing_bucket, because we don't actually want to collect docs that are missing the timestamp.

This, however, breaks when the index is entirely empty (even if the mapping has the timestamp field), or if all the documents in the index are lacking the timestamp.

We could fix this by setting missing_bucket: true on the date_histogram, and then adding an exists(timestamp) filter to the composite's query.

/cc @jen-huang

[elasticmachine]

Pinging @elastic/es-search-aggs

[jimczi]

This, however, breaks when the index is entirely empty (even if the mapping has the timestamp field), or if all the documents in the index are lacking the timestamp.

The error is thrown only if the field is unmapped so I guess that it happened because logstash-2015.05.21 was created without the utc_time field.

We could fix this by setting missing_bucket: true on the date_histogram, and then adding an exists(timestamp) filter to the composite's query.

This would disable the optimization in the composite aggregation when the query matches all documents. However I don't think we should throw an error if the field is unmapped so I opened #35331 to remove this limitation. It was there because such query is guaranteed to never create any bucket but this is not the expected behavior. We should just return an empty response in that case.