Skip to content

[Docs] Add profile section to the Search API documentation #25880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 26, 2017
Merged

[Docs] Add profile section to the Search API documentation #25880

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 80 additions & 5 deletions ...igh-level/src/test/java/org/elasticsearch/client/documentation/SearchDocumentationIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.ClearScrollRequest;
import org.elasticsearch.action.search.ClearScrollResponse;
import org.elasticsearch.action.search.SearchRequest;
Expand Down Expand Up @@ -52,6 +53,11 @@
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.profile.ProfileResult;
import org.elasticsearch.search.profile.ProfileShardResult;
import org.elasticsearch.search.profile.aggregation.AggregationProfileShardResult;
import org.elasticsearch.search.profile.query.CollectorResult;
import org.elasticsearch.search.profile.query.QueryProfileShardResult;
import org.elasticsearch.search.sort.FieldSortBuilder;
import org.elasticsearch.search.sort.ScoreSortBuilder;
import org.elasticsearch.search.sort.SortOrder;
Expand All @@ -70,6 +76,7 @@

import static org.elasticsearch.index.query.QueryBuilders.matchQuery;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.greaterThan;

/**
Expand Down Expand Up @@ -105,7 +112,7 @@ public void testSearch() throws IOException {
Arrays.asList("kimchy", "tanguy"), "innerObject", Collections.singletonMap("key", "value")));
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
BulkResponse bulkResponse = client.bulk(request);
assertSame(bulkResponse.status(), RestStatus.OK);
assertSame(RestStatus.OK, bulkResponse.status());
assertFalse(bulkResponse.hasFailures());
}
{
Expand Down Expand Up @@ -244,7 +251,7 @@ public void testSearchRequestAggregations() throws IOException {
.source(XContentType.JSON, "company", "Elastic", "age", 40));
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
BulkResponse bulkResponse = client.bulk(request);
assertSame(bulkResponse.status(), RestStatus.OK);
assertSame(RestStatus.OK, bulkResponse.status());
assertFalse(bulkResponse.hasFailures());
}
{
Expand Down Expand Up @@ -317,7 +324,7 @@ public void testSearchRequestSuggestions() throws IOException {
request.add(new IndexRequest("posts", "doc", "4").source(XContentType.JSON, "user", "cbuescher"));
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
BulkResponse bulkResponse = client.bulk(request);
assertSame(bulkResponse.status(), RestStatus.OK);
assertSame(RestStatus.OK, bulkResponse.status());
assertFalse(bulkResponse.hasFailures());
}
{
Expand Down Expand Up @@ -364,7 +371,7 @@ public void testSearchRequestHighlighting() throws IOException {
Arrays.asList("kimchy", "tanguy"), "innerObject", Collections.singletonMap("key", "value")));
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
BulkResponse bulkResponse = client.bulk(request);
assertSame(bulkResponse.status(), RestStatus.OK);
assertSame(RestStatus.OK, bulkResponse.status());
assertFalse(bulkResponse.hasFailures());
}
{
Expand Down Expand Up @@ -412,6 +419,74 @@ public void testSearchRequestHighlighting() throws IOException {
}
}

public void testSearchRequestProfiling() throws IOException {
RestHighLevelClient client = highLevelClient();
{
IndexRequest request = new IndexRequest("posts", "doc", "1")
.source(XContentType.JSON, "tags", "elasticsearch", "comments", 123);
request.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
IndexResponse indexResponse = client.index(request);
assertSame(RestStatus.CREATED, indexResponse.status());
}
{
SearchRequest searchRequest = new SearchRequest();
// tag::search-request-profiling
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.profile(true);
// end::search-request-profiling
searchSourceBuilder.query(QueryBuilders.termQuery("tags", "elasticsearch"));
searchSourceBuilder.aggregation(AggregationBuilders.histogram("by_comments").field("comments").interval(100));
searchRequest.source(searchSourceBuilder);

SearchResponse searchResponse = client.search(searchRequest);
// tag::search-request-profiling-get
Map<String, ProfileShardResult> profilingResults = searchResponse.getProfileResults(); // <1>
for (Map.Entry<String, ProfileShardResult> profilingResult : profilingResults.entrySet()) { // <2>
String key = profilingResult.getKey(); // <3>
ProfileShardResult profileShardResult = profilingResult.getValue(); // <4>
}
// end::search-request-profiling-get

ProfileShardResult profileShardResult = profilingResults.values().iterator().next();
assertNotNull(profileShardResult);

// tag::search-request-profiling-queries
List<QueryProfileShardResult> queryProfileShardResults = profileShardResult.getQueryProfileResults(); // <1>
for (QueryProfileShardResult queryProfileResult : queryProfileShardResults) { // <2>

}
// end::search-request-profiling-queries
assertThat(queryProfileShardResults.size(), equalTo(1));

for (QueryProfileShardResult queryProfileResult : queryProfileShardResults) {
// tag::search-request-profiling-queries-results
for (ProfileResult profileResult : queryProfileResult.getQueryResults()) { // <1>
String queryName = profileResult.getQueryName(); // <2>
long queryTimeInMillis = profileResult.getTime(); // <3>
List<ProfileResult> profiledChildren = profileResult.getProfiledChildren(); // <4>
}
// end::search-request-profiling-queries-results

// tag::search-request-profiling-queries-collectors
CollectorResult collectorResult = queryProfileResult.getCollectorResult(); // <1>
String collectorName = collectorResult.getName(); // <2>
Long collectorTimeInMillis = collectorResult.getTime(); // <3>
List<CollectorResult> profiledChildren = collectorResult.getProfiledChildren(); // <4>
// end::search-request-profiling-queries-collectors
}

// tag::search-request-profiling-aggs
AggregationProfileShardResult aggsProfileResults = profileShardResult.getAggregationProfileResults(); // <1>
for (ProfileResult profileResult : aggsProfileResults.getProfileResults()) { // <2>
String aggName = profileResult.getQueryName(); // <3>
long aggTimeInMillis = profileResult.getTime(); // <4>
List<ProfileResult> profiledChildren = profileResult.getProfiledChildren(); // <5>
}
// end::search-request-profiling-aggs
assertThat(aggsProfileResults.getProfileResults().size(), equalTo(1));
}
}

public void testScroll() throws IOException {
RestHighLevelClient client = highLevelClient();
{
Expand All @@ -424,7 +499,7 @@ public void testScroll() throws IOException {
.source(XContentType.JSON, "title", "The Future of Federated Search in Elasticsearch"));
request.setRefreshPolicy(WriteRequest.RefreshPolicy.IMMEDIATE);
BulkResponse bulkResponse = client.bulk(request);
assertSame(bulkResponse.status(), RestStatus.OK);
assertSame(RestStatus.OK, bulkResponse.status());
assertFalse(bulkResponse.hasFailures());
}
{
Expand Down
87 changes: 87 additions & 0 deletions docs/java-rest/high-level/apis/search.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,19 @@ the text `kmichy`
We will later see how to <<java-rest-high-retrieve-suggestions,retrieve suggestions>> from the
`SearchResponse`.

===== Profiling Queries and Aggregations

The {ref}/search-profile.html[Profile API] can be used to profile the execution of queries and aggregations for
a specific search request. in order to use it, the profile flag must be set to true on the `SearchSourceBuilder`:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling]
--------------------------------------------------

Once the `SearchRequest` is executed the corresponding `SearchResponse` will
<<java-rest-high-retrieve-profile-results,contain the profiling results>>.

[[java-rest-high-document-search-sync]]
==== Synchronous Execution

Expand Down Expand Up @@ -339,3 +352,77 @@ include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-suggestion
type of Suggestion class (here `TermSuggestion`), otherwise a `ClassCastException` is thrown
<3> Iterate over the suggestion entries
<4> Iterate over the options in one entry

[[java-rest-high-retrieve-profile-results]]
===== Retrieving Profiling Results

Profiling results are retrieved from a `SearchResponse` using the `getProfileResults()` method. This
method returns a `Map` containing a `ProfileShardResult` object for every shard involved in the
`SearchRequest` execution. `ProfileShardResult` are stored in the `Map` using a key that uniquely
identifies the shard the profile result corresponds to.

Here is a sample code that shows how to iterate over all the profiling results of every shard:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-get]
--------------------------------------------------
<1> Retrieve the `Map` of `ProfileShardResult` from the `SearchResponse`
<2> Profiling results can be retrieved by shard's key if the key is known, otherwise it might be simpler
to iterate over all the profiling results
<3> Retrieve the key that identifies which shard the `ProfileShardResult` belongs to
<4> Retrieve the `ProfileShardResult` for the given shard

The `ProfileShardResult` object itself contains one or more query profile results, one for each query
executed against the underlying Lucene index:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries]
--------------------------------------------------
<1> Retrieve the list of `QueryProfileShardResult`
<2> Iterate over each `QueryProfileShardResult`

Each `QueryProfileShardResult` gives access to the detailed query tree execution, returned as a list of
`ProfileResult` objects:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries-results]
--------------------------------------------------
<1> Iterate over the profile results
<2> Retrieve the name of the Lucene query
<3> Retrieve the time in millis spent executing the Lucene query
<4> Retrieve the profile results for the sub-queries (if any)

The Rest API documentation contains more information about {ref}/_profiling_queries.html[Profiling Queries] with
a description of the {ref}/_profiling_queries.html#_literal_query_literal_section[query profiling information]

The `QueryProfileShardResult` also gives access to the profiling information for the Lucene collectors:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries-collectors]
--------------------------------------------------
<1> Retrieve the profiling result of the Lucene collector
<2> Retrieve the name of the Lucene collector
<3> Retrieve the time in millis spent executing the Lucene collector
<4> Retrieve the profile results for the sub-collectors (if any)

The Rest API documentation contains more information about profiling information
{ref}/_profiling_queries.html#_literal_collectors_literal_section[for Lucene collectors].

In a very similar manner to the query tree execution, the `QueryProfileShardResult` objects gives access
to the detailed aggregations tree execution:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-aggs]
--------------------------------------------------
<1> Retrieve the `AggregationProfileShardResult`
<2> Iterate over the aggregation profile results
<3> Retrieve the type of the aggregation (corresponds to Java class used to execute the aggregation)
<4> Retrieve the time in millis spent executing the Lucene collector
<5> Retrieve the profile results for the sub-aggregations (if any)

The Rest API documentation contains more information about {ref}/_profiling_aggregations.html[Profiling Aggregations]