Painless Endpoint to Execute Scripts

[simianhacker]

This is a feature request to add an endpoint for Painless to execute arbitrary scripts. I'm imaging something like the following:

POST /_painless/execute
{
  "script": {
      "source": "params.count / params.total"
      "params": {
          "count": 100
          "total": 1000
      }
  }

and the response would be something like

{
   "result": 0.1
}

Obviously this is a very simple example but with this type of interface we could replace the "math" (which uses Math.js) aggregation in TSVB with Painless and provide a way for the user to create scripts that allow them to manipulate data essentially the same way (but more secure). This would also provide a way to create interfaces for and testing and debugging painless scripts. I'm sure Rashid could use this as well in the new Canvas project to provide scripting as well.

@jdconrad @rashidkpc @epixa

[jdconrad]

I'd also like to hear @rjernst 's thoughts on this too.

[rjernst]

This is a little bit complicated since each place in ES you use a script (called "context"s) provides different predefined variables. So I'm not sure it could be used for general painless script debugging because you can't just pass in any script from any part of ES. However, since we do now have contexts, what you are requesting should be possible. It would be a new context that doesn't provided any variables except params. The only caveat is we need to decide what type this context returns. I think it should be String. Would that work for you (you could parse the string back into a number if you expect a number for a particular script)?

[simianhacker]

@rjernst What are the different return types?

[jdconrad]

@simianhacker Any primitive or Object (which includes String), but since this is being sent as a REST request, I agree with @rjernst that a String makes sense to return since you can parse this as whatever you want on the client side, and it's simple to do.

[simianhacker]

Is there a simple way to take an array of results and encode them as a JSON string in Painless? then return that?

[jdconrad]

Not at this very moment, but with contexts we should be able to add a way to do this and give you useful methods for JSON -> String conversion.

[alexfrancoeur]

I'd like to weigh in here a bit from a product standpoint. The Math Agg @simianhacker created for 6.1 filled a huge gap that we have in Kibana. We have a number of users looking to do simple calculations such as dividing two metrics from within a Kibana visualization. There are of course more advanced users looking to create much more complex calculations. Bucket script support in Kibana has gotten us pretty far, but the ability to create free range painless scripts and calculations seamlessly within a visualization will provide significant value to an array of users utilizing our UI. I know 6.1.1 will be disappointing to a number of users who were waiting on TSVB's Math agg. If we can provide a more secure approach and similar experience native to elasticsearch - it will go a long way with our community. Just wanted to add my 2 cents here 😄

[rjernst]

@alexfrancoeur I'm not sure exactly what you are suggesting. What was proposed here is an independent endpoint to run painless scripts. But to run such a script with information from results of a given query would mean a second roundtrip. It means the "client side" processing of aggs results becomes not really client side (and probably painfully slow).

[simianhacker]

Yes, it would be a second round trip... painfully slow really depends on the latency and number of round trips required to do the calculation. If it was all batched together then it might be an additional 10ms to 60ms, which isn't terrible.

[jdconrad]

Longer term we probably want this be able to run against test docs.

[jdconrad]

@martijnvg Is this something that you're actively or planning to work on? Just don't want to double up work :)