Engine: store maxUnsafeAutoIdTimestamp in commit #24149
Conversation
The `maxUnsafeAutoIdTimestamp` time stamp is a safety marker guaranteeing that no indexing operation with a higher auto gen id timestamp was process by the engine. This allows us to safely process documents without checking if they were seen before. Currently this property is maintained in memory and is handed off from the primary to any replica during the recovery process. This PR takes a more natural approach and stores it in the lucene commit, using the same semantics (no op with a higher time stamp is part of this commit). This means that the knowledge is transferred during the file copy and also means that we don't need to worry about crazy situations where an original append only request arrives at the engine after a retry was processed *and* the engine was restarted.
bleskes
added
:Engine
:Distributed Indexing/Recovery
question, this won't be in all 5.x indices since they might do a full cluster restart? So we can't remove it since we might need it during recover process? The other option is to detect this on open but then we need to do this for all 5.x indices I think |
s1monw
left a comment
s1monw
left a comment
There was a problem hiding this comment.
I think it's a good change... left a suggestion
| } | ||
|
|
||
| private void updateMaxUnsafeAutoIdTimestampFromWriter(IndexWriter writer) { | ||
| long commitMaxUnsafeAutoIdTimestamp = Long.MIN_VALUE; |
There was a problem hiding this comment.
what's wrong with writer.getLiveCommitData().getOrDefault(MAX_UNSAFE_AUTO_ID_TIMESTAMP_COMMIT_ID, Long.MIN_VALUE);?
There was a problem hiding this comment.
it's an iterable :
public final synchronized Iterable<Map.Entry<String,String>> getLiveCommitData() {
I can throw it into a HashMap, if you prefer?
There was a problem hiding this comment.
I wondered if there was something better than iterating too but there's not since IndexWriter#getLiveCommitData only returns an Iterable.
There was a problem hiding this comment.
oh yeah oh well :) fair enough...
Agree it won't be there. However, on full cluster restart the primary is already opened with the maxUnsafeAutoIdTimestamp set to -1 and it flushes on start. This means that the future replicas will receive it when they do a file sync, so I think we can still remove this from the peer recovery code? |
fair enough - lets put a comment in the code pls |
|
Thx @s1monw & @jasontedor |
The `maxUnsafeAutoIdTimestamp` timestamp is a safety marker guaranteeing that no retried-indexing operation with a higher auto gen id timestamp was process by the engine. This allows us to safely process documents without checking if they were seen before. Currently this property is maintained in memory and is handed off from the primary to any replica during the recovery process. This commit takes a more natural approach and stores it in the lucene commit, using the same semantics (no retry op with a higher time stamp is part of this commit). This means that the knowledge is transferred during the file copy and also means that we don't need to worry about crazy situations where an original append only request arrives at the engine after a retry was processed *and* the engine was restarted.
With #24149 , it is now stored in the Lucene commit and is implicitly transferred in the file phase of the recovery.
With elastic#24149 , it is now stored in the Lucene commit and is implicitly transferred in the file phase of the recovery.
…ID if index was created before 5.5.0 It was adding in #24149 which was merged into 5.5.0
…ID if index was created before 5.5.0 It was adding in #24149 which was merged into 5.5.0
clintongormley
added
:Distributed Indexing/Distributed
4 participants
The
maxUnsafeAutoIdTimestamptimestamp is a safety marker guaranteeing that no retried-indexing operation with a higher auto gen id timestamp was process by the engine. This allows us to safely process documents without checking if they were seen before.Currently this property is maintained in memory and is handed off from the primary to any replica during the recovery process.
This PR takes a more natural approach and stores it in the lucene commit, using the same semantics (no retry op with a higher time stamp is part of this commit). This means that the knowledge is transferred during the file copy and also means that we don't need to worry about crazy situations where an original append only request arrives at the engine after a retry was processed and the engine was restarted.
Once this is in 5.x, I will submit a follow up PR to remove this part of the recovery logic.