Add cache or not details to profile api output

[ppf2]

Elasticsearch version: 2.3

Describe the feature:

While the query cache in 2.x makes decisions automatically on what to cache and what not to cache, it does not rewrite queries automatically to improve cache-ability. It will be useful to include caching details as part of the profile api output to indicate which clauses are being cached vs. not cached. This will be helpful additional diagnostic information to have, for some of queries with not cached clauses may not end up showing up in the hot threads (and there is not a way to examine the cached entries or the entires that were part of the history of entries).

[makeyang]

+1

[polyfractal]

I just had a skim through the code to see how difficult this would be. I think it'll be relatively non-trivial. The cache is instantiated as a shared, per-node resource (even though it tracks queries on a per-shard basis). So we can't "wrap" the cache with an instrumented profiler like we can with queries, since it will be concurrently servicing non-profiled queries too.

The API is a bit tricky too; caching the queries is a side-effect of creating the weight... the caller doesn't know if the filter was actually cached or not.

There may be a way to wrap the CachingWeightWrapper with an instrumented profiler, which immediately asks for the various cache stats (size, hitCount, etc) and uses that to approximate if the filter was cached or not. It'd only be a guess though. Other, concurrent queries could have been caching at the same time in unrelated non-profiled queries.

Dunno. I'll corner @jpountz in Prague and see what he thinks :)

[sundarv85]

+1

This is definitely a very valuable feature esp when debugging complex caching issues. Currently in our stack, we have indexes with 200 Million documents and aggregation response takes somewhere between 4-60 seconds. Debugging such issue is too complicated. When profile API can respond back if the response comes from cache or not, such information could be very helpful.

[elasticmachine]

Pinging @elastic/es-search-aggs

[ppf2]

Starting in 7.6, profile API now disables the query and request cache when
profile is set to true (#48195) so users will not have to deal with figuring out if the profile API output is reflecting query performance with/without request/query caching in play, etc..