Allow Bucket Script aggregation to reference on string results

[larrycinnabar]

Currently bucket script aggregations allows only numbers to be referenced on:

      "some-stringified-metric":{  
          "bucket_script":{  
            "buckets_path":{  
              "a":"path>of>nested>metrics>agg_that_returns_a_string.value"
            },
            "script":{  
              "source":"params.a"
            }
          }
        },

You will get:
"buckets_path must reference either a number value or a single value numeric metric aggregation, got: java.lang.String"

Result of the agg is just a string, not a list of object, or something complex. Why plain single strings are so offended that they can not be returned as a result?

[elasticmachine]

Pinging @elastic/es-analytics-geo

[polyfractal]

Right now, most of the pipeline aggs are geared towards doing numeric transformations (derivatives, etc), which is where that limitation is coming from. The BucketScript agg is definitely a little different, since you have access to a script.

What sort of operation are you wanting to do with the strings?

[larrycinnabar]

My case is a little specific:

I have a query with dozens of aggregations. Some of them are really nested. Then, on application level we need to read elastic raw response (json) and marshal it to a struct. To simplify this thing we did a trick: For every metric that we need - we will have:

1. real aggregation (may be really deep nested)
2. "public" aggregation - that is top-level and just uses bucket script to direct to its real aggregation value

Why strings are needed? - Because some metrics can return not numeric result, but the value result .
I provide here two examples:

1. Simple example: the most used term
2. Complex example. Some of our metrics - are histograms: [{key:a,count:100},{key:b,count:200}...]. It's might be a result of terms aggregation, but we use ScriptedMetric: because sometimes (a) logic is behind just a TermsAggregation, and (b) we can jsonify the result as a simple string.
   So, yes, I want to perform a some kind of terms aggregation, and the list of buckets it returns - are jsonified in a string, and I want it to be available as top-level aggregation (via bucket-script)

[polyfractal]

I see, thanks for the detailed description! I think it makes sense to open up BucketScript for any kind of return value, not just numerics, given the open-ended nature of scripts. It will still need to support gap_policy and related numeric-based features (which may not make sense for strings), but I think that's probably acceptable.

I think it would be a pretty straightforward change: modify the BucketScript painless context to return Object instead of Double, adjust the aggregator to work with objects, update the docs.

I'm going to label this team-discuss to see what the rest of the team thinks.

[polyfractal]

Discussed this in the team meeting, and there was no objection to bucket_script being able to return Object instead of doubles. Unfortunately, I think implementing this will probably be tied up with larger refactoring done to the pipeline framework, so it may be some time before something like this can be extended/fixed.

[Hohol]

I'd like to work on this.