Add fromXContent method to SearchResponse #24720
Conversation
SearchResponse#fromXContent allows to parse a search response, including search hits, aggregations, suggestions and profile results. Only the aggs that we can parse today are supported (which means all of them but a couple that are left to support). SearchResponseTests reuses the existing test infra to randomize aggregations, suggestions and profile response. Relates to elastic#23331
javanna
force-pushed
the
enhancement/search_response_parsing
branch
from
1a46f86 to
b8ad84e
Compare
|
|
||
| public static SearchResponse fromXContent(XContentParser parser) throws IOException { | ||
| XContentParser.Token token; | ||
| ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.currentToken(), parser::getTokenLocation); |
There was a problem hiding this comment.
nit: can you move the ensureExpectedToken() first?
| XContentParserUtils.ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser::getTokenLocation); | ||
| SearchResponse parsed = SearchResponse.fromXContent(parser); | ||
| // check that we at least get the same number of shardFailures | ||
| assertEquals(response.getShardFailures().length, parsed.getShardFailures().length); |
There was a problem hiding this comment.
We have some tests that create a failure instance and its expected instance, maybe we could have something similar here and use ElasticsearchExceptionTests.randomExceptions()?
| boolean humanReadable = randomBoolean(); | ||
| final ToXContent.Params params = new ToXContent.MapParams(singletonMap(RestSearchAction.TYPED_KEYS_PARAM, "true")); | ||
| BytesReference originalBytes = toShuffledXContent(response, xcontentType, params, humanReadable); | ||
| XContentParser parser = createParser(xcontentType.xContent(), originalBytes); |
There was a problem hiding this comment.
Can you use try-with-resources here so that resources are released asap?
|
@tlrx can you please check if my latest commit addresses your concerns? |
| } else if (NUM_REDUCE_PHASES.match(currentFieldName)) { | ||
| numReducePhases = parser.intValue(); | ||
| } else { | ||
| throwUnknownField(currentFieldName, parser.getTokenLocation()); |
There was a problem hiding this comment.
We started being lenient with other parts of the response in order to allow new fields in the future to work with an older client. Should we also do this on this level? This applies to all similar checks in this PR.
There was a problem hiding this comment.
I think we are free to do whatever we want to do until we test that leniency. I went for using throwUnknownField as that one may one day conditionally throw. Or maybe we have to remove it. But once we test this behaviour, we will have to go back to each parsing method and check that we do the right thing
| + "\"max_score\":1.5," | ||
| + "\"hits\":[{\"_type\":\"type\",\"_id\":\"id1\",\"_score\":2.0}]" | ||
| + "}" | ||
| + "}", Strings.toString(response)); |
There was a problem hiding this comment.
Although I liked it at first, I'm starting to doubt the usefulness of these tests. And formatting makes it unreadable. If we keep it, can you change this to something like:
StringBuilder expectedString = new StringBuilder();
expectedString.append("{");
{
expectedString.append("\"took\":0,");
expectedString.append("\"timed_out\":false,");
expectedString.append("\"_shards\":");
{
expectedString.append("{\"total\":0,");
expectedString.append("\"successful\":0,");
expectedString.append("\"failed\":0},");
}
expectedString.append("\"hits\":");
{
expectedString.append("{\"total\":100,");
expectedString.append("\"max_score\":1.5,");
expectedString.append("\"hits\":[{\"_type\":\"type\",\"_id\":\"id1\",\"_score\":2.0}]}");
}
}
expectedString.append("}");
assertEquals(expectedString.toString(), Strings.toString(response));
That should be more robust against auto-formatting.
There was a problem hiding this comment.
I don't find it much more readable when using a string builder though :)
There was a problem hiding this comment.
Maybe not much, but not even a tiny bit better?
There was a problem hiding this comment.
Just a suggestion, I don't mind if we remove it here entirely. I used to like those tests because they show how a typical xContent output of those classes looks like. But with these large ones its kind of debatable whether it is useful.
There was a problem hiding this comment.
I reformatted, was just a copy paste, thanks for doing it ;)
SearchResponse#fromXContent allows to parse a search response, including search hits, aggregations, suggestions and profile results. Only the aggs that we can parse today are supported (which means all of them but a couple that are left to support). SearchResponseTests reuses the existing test infra to randomize aggregations, suggestions and profile response. Relates to elastic#23331
3 participants
SearchResponse#fromXContent allows to parse a search response, including search hits, aggregations, suggestions and profile results. Only the aggs that we can parse today are supported (which means all of them but a couple that are left to support). SearchResponseTests reuses the existing test infra to randomize aggregations, suggestions and profile response.
This PR contains some of the changes proposed with #22533 and add extensive tests for it based on the existing test infra for parsing responses.
Relates to #23331