[ML] Support rolled-up indices in datafeeds

[dimitris-athanasiou]

As users start using roll-ups to downsize their indices, it makes sense for datafeeds to be able to fetch data from rolled-up indices.

[elasticmachine]

Pinging @elastic/ml-core

[dimitris-athanasiou]

A quick look at that revealed that we cannot scroll using rollup-search. I think this means we can only support rolled-up indices when using aggregations.

Also, it is worth noting that instead of using field-caps, for rollups we should make use of the rollup-caps API to check if the analyzed fields are available.

Another question is how we handle index patterns that match both raw and rollup indices.

[droberts195]

Notes from our discussion about how to implement this:

• No datafeed config changes are required
• We will use rollup search if the rollup caps action finds any rollup indices in the indices list in the datafeed config
• We will throw an error if rollup caps finds any rolled up indices and the datafeed config does not contain aggregations or if the datafeed date histogram interval is not a multiple of the rollup date histogram interval
• We will throw an error if rollup caps finds any rolled up indices and they cannot service the datafeed's aggregation
• We will create an auditor notification when it is decided that a datafeed will use rollups
• Make sure the preview endpoint works for rollups

[benwtrent]

This issue was closed with PR: #34654