Skip to content

Conversation

@dnhatn
Copy link
Member

@dnhatn dnhatn commented Nov 14, 2019

With peer recovery retention leases and sequence-number based replica allocation, a regular flush can speed up recovery as a synced-flush. With this change, we will flush instead of synced-flush when a shard becomes inactive.

Depends on #48430
Closes #31965

@dnhatn dnhatn added >enhancement :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. v8.0.0 v7.6.0 labels Nov 14, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Distributed)

Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will still need the old sync-flush logic in 7.x to handle 6.x/7.x indices without soft-deletes as rolling/full cluster restarts are o.w. going to be painful.

@dnhatn dnhatn removed the v7.6.0 label Nov 19, 2019
@dnhatn
Copy link
Member Author

dnhatn commented Nov 19, 2019

I think we will still need the old sync-flush logic in 7.x to handle 6.x/7.x indices without soft-deletes as rolling/full cluster restarts are o.w. going to be painful.

Yes, we should keep this logic in 7.x.

Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I think the flushing behavior should be backported to 7.x (and the sync-flush behavior kept in 7.x as well for non-soft-deletes indices)

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dnhatn
Copy link
Member Author

dnhatn commented Nov 22, 2019

Thanks @ywelsch and @henningandersen for reviewing.

@dnhatn dnhatn merged commit 725dda3 into elastic:master Nov 22, 2019
@dnhatn dnhatn deleted the flush-on-inactive branch November 22, 2019 03:25
dnhatn added a commit that referenced this pull request Jan 23, 2020
If all nodes are on 7.6, we prefer to perform a normal flush 
instead of synced flush when a shard becomes inactive.

Backport of #49126
@dnhatn dnhatn added the v7.7.0 label Jan 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. >enhancement v7.7.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flush inactive shards

5 participants