Lift retention lease expiration to index shard #38380
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit lifts the control of when retention leases are expired to index shard. In this case, we move expiration to an explicit action rather than a side-effect of calling ReplicationTracker#getRetentionLeases. This explicit action is invoked on a timer. If any retention leases expire, then we hard sync the retention leases to the replicas. Otherwise, we proceed with a background sync.
jasontedor
added
>enhancement
v7.0.0
:Distributed Indexing/Distributed
Collaborator
|
Pinging @elastic/es-distributed |
dnhatn
approved these changes
Feb 4, 2019
Member
dnhatn
left a comment
dnhatn
left a comment
There was a problem hiding this comment.
This makes sense to me. LGTM. How about lifting onAddRetentionLease to IndexShard?
server/src/main/java/org/elasticsearch/index/seqno/ReplicationTracker.java
Outdated
Show resolved
Hide resolved
…-lease-expiration * elastic/master: (24 commits) Add support for API keys to access Elasticsearch (elastic#38291) Add typless client side GetIndexRequest calls and response class (elastic#37778) Limit token expiry to 1 hour maximum (elastic#38244) add docs saying mixed-cluster ILM is not supported (elastic#37954) Skip unsupported languages for tests (elastic#38328) Deprecate `_type` in simulate pipeline requests (elastic#37949) Mute testCannotShrinkLeaderIndex (elastic#38374) Tighten mapping syncing in ccr remote restore (elastic#38071) Add test for `PutFollowAction` on a closed index (elastic#38236) Fix SSLContext pinning to TLSV1.2 in reload tests (elastic#38341) Mute RareClusterStateIT.testDelayedMappingPropagationOnReplica (elastic#38357) Deprecate types in rollover index API (elastic#38039) Types removal - fix FullClusterRestartIT warning expectations (elastic#38310) Fix ILM explain response to allow unknown fields (elastic#38054) Mute testFollowIndexAndCloseNode (elastic#38360) Docs: Drop inline callout from scroll example (elastic#38340) Deprecate HLRC security methods (elastic#37883) Remove types from Monitoring plugin "backend" code (elastic#37745) Add Composite to AggregationBuilders (elastic#38207) Clarify slow cluster-state log messages (elastic#38302) ...
Member
Author
+1 in a follow-up. |
This was referenced Feb 5, 2019
Member
Author
|
@elasticmachine test this please |
Member
Author
|
@elasticmachine run elasticsearch-ci/2 |
* master: Mute failing API key integration test (elastic#38409) Change the milliseconds precision to 3 digits for intervals. (elastic#38297) SecuritySettingsSource license.self_generated: trial (elastic#38233) Rename no-master-block setting (elastic#38350) Rename static Zen1 settings (elastic#38333) Migration doc for audit json log file (elastic#38165) Add apm_user reserved role (elastic#38206)
Member
Author
|
@elasticmachine test this please |
Member
Author
|
@elasticmachine run elasticsearch-ci/default-distro |
Member
Author
|
@elasticmachine run elasticsearch-ci/2 |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 5, 2019
* master: (23 commits) Lift retention lease expiration to index shard (elastic#38380) Make Ccr recovery file chunk size configurable (elastic#38370) Prevent CCR recovery from missing documents (elastic#38237) re-enables awaitsfixed datemath tests (elastic#38376) Types removal fix FullClusterRestartIT warnings (elastic#38445) Make sure to reject mappings with type _doc when include_type_name is false. (elastic#38270) Updates the grok patterns to be consistent with logstash (elastic#27181) Ignore type-removal warnings in XPackRestTestHelper (elastic#38431) testHlrcFromXContent() should respect assertToXContentEquivalence() (elastic#38232) add basic REST test for geohash_grid (elastic#37996) Remove DiscoveryPlugin#getDiscoveryTypes (elastic#38414) Fix the clock resolution to millis in GetWatchResponseTests (elastic#38405) Throw AssertionError when no master (elastic#38432) `if_seq_no` and `if_primary_term` parameters aren't wired correctly in REST Client's CRUD API (elastic#38411) Enable CronEvalToolTest.testEnsureDateIsShownInRootLocale (elastic#38394) Fix failures in BulkProcessorIT#testGlobalParametersAndBulkProcessor. (elastic#38129) SQL: Implement CURRENT_DATE (elastic#38175) Mute testReadRequestsReturnLatestMappingVersion (elastic#38438) [ML] Report index unavailable instead of waiting for lazy node (elastic#38423) Update Rollup Caps to allow unknown fields (elastic#38339) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit lifts the control of when retention leases are expired to index shard. In this case, we move expiration to an explicit action rather than a side-effect of calling
ReplicationTracker#getRetentionLeases. This explicit action is invoked on a timer. If any retention leases expire, then we hard sync the retention leases to the replicas. Otherwise, we proceed with a background sync.Relates #37165
Closes #37963
Supersedes #38070