Add response_transform_script to allow xpath style selection of response elements #7401
Description
Our JSON responses are currently hard coded. If you only need one piece of information you need to retrieve, and parse, a fairly lengthy JSON document to get it. Although the browser is generally good at this, large responses create problems and require moving data over a, potentially slow, wire, only to throw it away immediately. It would be really nice to only get from the backend (Elasticsearch) the data that is required.
This issue has cropped up in a few places, eg #2149, #7350, #7330 each of which have slightly different requirements. Instead of supporting multiple options to turn certain parameters on or off, it makes sense to provide a single generic solution that is flexible enough to solve all of these problems.
The best solution that we have found is to use GPath:
GPath is a path expression language integrated into Groovy which allows parts of nested structured data to be identified. In this sense, it has similar aims and scope as XPath does for XML.
We will add a response_transform_script parameter on all APIs (in the body or in the query string) which provides the full JSON response as a GPath object (eg _response) which can be manipulated at will. For instance, to return just the _source fields from the hits array from a search request, you could do:
GET /_search
{
"response_transform_script: {
"script": "_json.hits.hits.collect { it._source }"
}
}