Boxplot Aggregation

[jtibshirani]

(Previous title: 'Interquartile Range Aggregation')

The interquartile range is a common robust measure of statistical dispersion. Compared to the standard deviation, the IQR is less sensitive to outliers in the data, with a breakdown point of 0.25. Along with the median, it is often used in creating a box plot, a simple yet common way to summarize data and identify potential outliers.

The IQR is equal to the third minus the first quartile of a dataset, and could be calculated from the output of a percentiles aggregation. Even though it can be easily calculated from quantile information, it may still be useful to provide it as an aggregation for convenience, and to increase its visibility. An alternative option would be to describe the IQR as part of the percentiles documentation.

Compared to the MAD (#26681), the IQR has a lower breakdown point (0.25, compared to 0.5). However, it is simple to calculate and is better equipped to handle skewed (asymmetric) data.

Relates to #26681.

[elasticmachine]

Pinging @elastic/es-search-aggs

[pmoust]

subscribe

[mattfield]

sub

[jtibshirani]

After more thought, I think it’d be useful to provide a boxplot aggregation that returns essential information for making a boxplot. The output could look something like the following:

{
    "min": 0.1,
    "q1": 0.15,
    "q2": 0.43,
    "q3": 0.5,
    "max": 0.67
} 

The IQR could then be easily calculated through q3 - q1 (we could also just include an iqr entry in the response if we think it’d be convenient).

In a basic boxplot, the whiskers extend to the min and max values. But in another popular style, the whiskers extend to the furthest points within [Q1 - 1.5 * IQR, Q3 + 1.5 * IQR]. Points that are outside this interval are considered outliers and usually displayed on the plot. Perhaps we could start with the basic five-number summary, then look into supporting the style with outliers if there’s interest.