Add a new auto_histogram aggregation for numeric fields

[pcsanwald]

Per #28993, we also want to add support for an auto_histogram on numeric fields. We agreed that it should be a separate issue so that we would not block #28993 on implementation.

[elasticmachine]

Pinging @elastic/es-search-aggs

[melissachang]

Here's a suggestion, what do you think?

It's easier for the user if buckets are round numbers, eg:

10-19:     25
20-29      86
30-39      83

is easier to parse than:

10-17:     25
18-25      86
26-33      83

So I propose:
bucket is optional.

• If bucket is set, that number of buckets is returned.
• If bucket is not set, the number of buckets returned will vary, to make the buckets round. The number of buckets returned can vary between 5 and 12.

Examples:

• Min value = 1, max_value = 20. 10 buckets: 1-2, 3-4, 5-6, 7-8, 9-10, 11-12, ...
• Min value = 1, max value = 50. 6 buckets: 0-9, 10-19, 20-29, 30-39, 40-49, 50-59
• Min value = 1, max value = 99. 10 buckets: 0-9, 10-19, 20-29, 30-39, ..., 90-99
• Min value = 1, max value = 100. 11 buckets: 0-9, 10-19, 20-29, 30-39, ..., 90-99, 100-109
• Min value = 200, max value = 300. 11 buckets: 200-209, 210-219, 220-229, 230-239, ..., 290-299, 200-209

FYI @rayward who filed #9572.

[colings86]

@melissachang what you are describing about avoiding "non-round" intervals is actually what we intend to do and what is already done for the date version of this aggregation in #28993. The difference is that the intervals are always selected to be "round". the buckets parameter the user provides represents the maximum number of buckets they would like returned and the aggregation will pick a "round" interval which gets as close to that number as possible but does not exceed it.

This means that an application can, for example, provide the maximum number of bars that it has space to render on a bar chart, and the aggregation will return buckets with an interval easy for humans to parse but which do not exceed the number the application is able to render.

[melissachang]

I see, thanks.

The documentation for #28993 says "The buckets field is optional, and will default to 10 buckets if not specified.". Does "rounding" occur in this case? For me, ideal is:

• I would like to use this feature without specifying max # buckets. (I don't want to think about how many buckets I want, I just want a reasonable default.)
• When I don't specify max # buckets, rounding takes place and the actual # buckets may vary.

[colings86]

Yes rounding also occurs for the default value, so you may get less than 10 buckets returned by default but not more than 10.

[melissachang]

Thanks. Is there an ETA on when this will be implemented? I would like to start using it. :)

[colings86]

We don't have an ETA for this sorry but its great that you are excited for this feature. You can track this issue for progress on the feature, where there is a PR available it will show the intended version(s) (note until the PR is merged the intended versions may change)

[melissachang]

Thanks. If I could test the future PR before it's merged, that would be great. I'd like to see how it looks on my data.

[mrec]

Presumably this would also follow auto_date_histogram's policy of erroring on buckets above 10k?

One (very) edge case worth considering is what happens when "buckets":1. This is obviously silly for auto_histogram, but it's not silly for other bucketed aggs like terms so I don't know if you'd want to error on it. If you don't error, though, you've got the awkward question of what to do when the value range crosses zero - normally for a distribution like that zero would always be a bucket boundary, but in this case it couldn't be.

Finally, could you confirm that your rounding logic will always round to 1x, 2x or 5x a power of ten? I'm building a manual equivalent of this feature at the moment, using a separate calibration request, and am trying to design it in such a a way that we could swap it out for auto_histogram transparently when available.