Skip to content

Follow stats structure #34301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Oct 5, 2018
Merged

Follow stats structure #34301

merged 5 commits into from
Oct 5, 2018

Conversation

@jasontedor
Copy link
Member

@jasontedor jasontedor commented Oct 4, 2018

This commit modifies the follow stats API response structure to more clearly highlight meaning of the higher level fields. In particular, previously the response had a top-level key for each index. Instead, we nest the indices under an "indices" field which is now an array. The values in this array are objects containing two fields: "index" which is the name of the follower index, and "shards" which is an array where each value in the array is the follower stats for that shard. That is, we have gone from:

{
  "bar": [
    {
      "shard_id": 0...
    }...
  ]...
}

to

{
  "indices": [
    {
      "index": "bar",
      "shards": [
        {
          "shard_id": 0...
        }...
      ]
   }...
}

@jasontedor
Rename CCR stats implementation
6474805 
In the CCR docs we want to refer to the endpoint that returns following
stats as the follow stats API. This commit renames the internal
implementation of this endpoint to reflect this usage.
@jasontedor
Adjust follow stats response structure
ae70db7 
This commit modifies the follow stats API response structure to more
clearly highlight meaning of the higher level fields. In particular,
previously the response had a top-level key for each index. Instead, we
nest the indices under an "indices" field which is now an array. The
values in this array are objects containing two fields: "index" which is
the name of the follower index, and "shards" which is an array where
each value in the array is the follower stats for that shard. That is,
we have gone from:

{
  "bar": [
    {
      "shard_id": 0...
    }...
  ]...
}

to

{
  "indices": [
    {
      "index": "bar",
      "shards": [
        {
          "shard_id": 0...
        }...
      ]
   }...
}
@jasontedor jasontedor added >non-issue review :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features labels Oct 4, 2018
@jasontedor jasontedor requested a review from martijnvg October 4, 2018 12:22
@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 4, 2018

Pinging @elastic/es-distributed

@jasontedor
Copy link
Member Author

jasontedor commented Oct 4, 2018

Note that this PR also contains the changes from #34300 so that I could open them in parallel. This PR will merge in master after #34300 is integrated, which will remove the changes from #34300 from this changeset.

@jasontedor
Copy link
Member Author

jasontedor commented Oct 4, 2018

The only two files actually changed by this PR are:

  • ccr/qa/rest/src/test/resources/rest-api-spec/test/ccr/follow_stats.yml
  • ccr/action/FollowStatsAction.java

@jasontedor
Merge branch 'master' into rename-ccr-stats
6833ad7 
* master: (25 commits)
  HLRC: ML Adding get datafeed stats API (elastic#34271)
  Small fixes to the HLRC watcher documentation. (elastic#34306)
  Tasks: Document that status is not semvered (elastic#34270)
  HLRC: ML Add preview datafeed api (elastic#34284)
  [CI] Fix bogus ScheduleWithFixedDelayTests.testRunnableRunsAtMostOnceAfterCancellation
  Fix error in documentation for activete watch
  SCRIPTING: Terms set query expression (elastic#33856)
  Logging: Drop remaining Settings log ctor (elastic#34149)
  [ML] Remove unused last_data_time member from Job (elastic#34262)
  Docs: Allow skipping response assertions (elastic#34240)
  HLRC: Add activate watch action (elastic#33988)
  [Security] Multi Index Expression alias wildcard exclusion (elastic#34144)
  [ML] Label anomalies with  multi_bucket_impact (elastic#34233)
  Document smtp.ssl.trust configuration option (elastic#34275)
  Support PKCS#11 tokens as keystores and truststores  (elastic#34063)
  Fix sporadic failure in NestedObjectMapperTests
  [Authz] Allow update settings action for system user (elastic#34030)
  Replace version with reader cache key in IndicesRequestCache (elastic#34189)
  [TESTS] Set SO_LINGER and SO_REUSEADDR on the mock socket (elastic#34211)
  Security: upgrade unboundid ldapsdk to 4.0.8 (elastic#34247)
  ...
@jasontedor
Merge branch 'rename-ccr-stats' into follow-stats-structure
11f4565 
* rename-ccr-stats: (25 commits)
  HLRC: ML Adding get datafeed stats API (elastic#34271)
  Small fixes to the HLRC watcher documentation. (elastic#34306)
  Tasks: Document that status is not semvered (elastic#34270)
  HLRC: ML Add preview datafeed api (elastic#34284)
  [CI] Fix bogus ScheduleWithFixedDelayTests.testRunnableRunsAtMostOnceAfterCancellation
  Fix error in documentation for activete watch
  SCRIPTING: Terms set query expression (elastic#33856)
  Logging: Drop remaining Settings log ctor (elastic#34149)
  [ML] Remove unused last_data_time member from Job (elastic#34262)
  Docs: Allow skipping response assertions (elastic#34240)
  HLRC: Add activate watch action (elastic#33988)
  [Security] Multi Index Expression alias wildcard exclusion (elastic#34144)
  [ML] Label anomalies with  multi_bucket_impact (elastic#34233)
  Document smtp.ssl.trust configuration option (elastic#34275)
  Support PKCS#11 tokens as keystores and truststores  (elastic#34063)
  Fix sporadic failure in NestedObjectMapperTests
  [Authz] Allow update settings action for system user (elastic#34030)
  Replace version with reader cache key in IndicesRequestCache (elastic#34189)
  [TESTS] Set SO_LINGER and SO_REUSEADDR on the mock socket (elastic#34211)
  Security: upgrade unboundid ldapsdk to 4.0.8 (elastic#34247)
  ...
martijnvg
martijnvg approved these changes Oct 5, 2018
View reviewed changes
Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jasontedor
Merge branch 'master' into follow-stats-structure
cdd96aa 
* master:
  Rename CCR stats implementation (elastic#34300)
  Add max_children limit to nested sort (elastic#33587)
  MINOR: Remove Dead Code from Netty4Transport (elastic#34134)
  Rename clsuterformation -> testclusters (elastic#34299)
  [Build] make sure there are no duplicate classes in third party audit (elastic#34213)
  BWC Build: Read CI properties to determine java version (elastic#34295)
  [DOCS] Fix typo and add [float]
  Allow User/Password realms to disable authc (elastic#34033)
  Enable security automaton caching (elastic#34028)
  Preserve thread context during authentication. (elastic#34290)
  [ML] Allow asynchronous job deletion (elastic#34058)
@jasontedor jasontedor merged commit 7d57bdb into elastic:master Oct 5, 2018
jasontedor added a commit that referenced this pull request Oct 5, 2018
@jasontedor
Follow stats structure (#34301)
6252301 
This commit modifies the follow stats API response structure to more
clearly highlight meaning of the higher level fields. In particular,
previously the response had a top-level key for each index. Instead, we
nest the indices under an "indices" field which is now an array. The
values in this array are objects containing two fields: "index" which is
the name of the follower index, and "shards" which is an array where
each value in the array is the follower stats for that shard. That is,
we have gone from:

{
  "bar": [
    {
      "shard_id": 0...
    }...
  ]...
}

to

{
  "indices": [
    {
      "index": "bar",
      "shards": [
        {
          "shard_id": 0...
        }...
      ]
   }...
}
@jasontedor jasontedor deleted the follow-stats-structure branch October 5, 2018 10:43
jasontedor added a commit to martijnvg/elasticsearch that referenced this pull request Oct 5, 2018
@jasontedor
Merge branch 'master' into ccr_clear_fetch_errors_empty_fetch_response
69c3484 
* master: (63 commits)
  [Build] randomizedtesting: Allow property values to be closures (elastic#34319)
  Feature/hlrc ml docs cleanup (elastic#34316)
  Docs: DRY up CRUD docs (elastic#34203)
  Minor corrections in geo-queries.asciidoc (elastic#34314)
  [DOCS] Remove beta label from normalizers (elastic#34326)
  Adjust size of BigArrays in circuit breaker test
  Adapt bwc version after backport
  Follow stats structure (elastic#34301)
  Rename CCR stats implementation (elastic#34300)
  Add max_children limit to nested sort (elastic#33587)
  MINOR: Remove Dead Code from Netty4Transport (elastic#34134)
  Rename clsuterformation -> testclusters (elastic#34299)
  [Build] make sure there are no duplicate classes in third party audit (elastic#34213)
  BWC Build: Read CI properties to determine java version (elastic#34295)
  [DOCS] Fix typo and add [float]
  Allow User/Password realms to disable authc (elastic#34033)
  Enable security automaton caching (elastic#34028)
  Preserve thread context during authentication. (elastic#34290)
  [ML] Allow asynchronous job deletion (elastic#34058)
  HLRC: ML Adding get datafeed stats API (elastic#34271)
  ...
kcm pushed a commit that referenced this pull request Oct 30, 2018
@jasontedor @kcm
Follow stats structure (#34301)
6bbcb33 
This commit modifies the follow stats API response structure to more
clearly highlight meaning of the higher level fields. In particular,
previously the response had a top-level key for each index. Instead, we
nest the indices under an "indices" field which is now an array. The
values in this array are objects containing two fields: "index" which is
the name of the follower index, and "shards" which is an array where
each value in the array is the follower stats for that shard. That is,
we have gone from:

{
  "bar": [
    {
      "shard_id": 0...
    }...
  ]...
}

to

{
  "indices": [
    {
      "index": "bar",
      "shards": [
        {
          "shard_id": 0...
        }...
      ]
   }...
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@martijnvg martijnvg martijnvg approved these changes

Assignees

No one assigned

Labels

:Distributed Indexing/CCR Issues around the Cross Cluster State Replication features >non-issue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jasontedor @elasticmachine @martijnvg