BlobContainer#writeBlob no longer can overwrite a blob #19749
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
conform with the requirements of the writeBlob method by throwing a FileAlreadyExistsException if attempting to write to a blob that already exists. This change means implementations of BlobContainer should never overwrite blobs - to overwrite a blob, it must first be deleted and then can be written again. Closes elastic#15579
abeyad
added
>enhancement
review
:Distributed Coordination/Snapshot/Restore
Contributor
|
LGTM. |
Author
|
thanks @dadoonet |
8 tasks
This was referenced May 7, 2018
ywelsch
added a commit
that referenced
this pull request
May 17, 2018
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist. Relates to #19749
ywelsch
added a commit
that referenced
this pull request
May 17, 2018
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist. Relates to #19749
ywelsch
added a commit
to ywelsch/elasticsearch
that referenced
this pull request
May 23, 2018
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides an option (with stronger consistency guarantees) on the insert method that guarantees that the blob that's uploaded does not already exist. Relates to elastic#19749
ywelsch
added a commit
that referenced
this pull request
Jun 6, 2018
In #19749 an extra check was added before writing each blob to ensure that we would not be overriding an existing blob. Due to S3's weak consistency model, this check was best effort. To make matters worse, however, this resulted in a HEAD request to be done before every PUT, in particular also when PUTTING a new object. The approach taken in #19749 worsened our consistency guarantees for follow-up snapshot actions, as it made it less likely for new files that had been written to be available for reads. This commit therefore removes this extra check. Due to the weak consistency model, this check was a best effort thing anyway, and there's currently no way to prevent accidental overrides on S3.
ywelsch
added a commit
that referenced
this pull request
Jun 6, 2018
In #19749 an extra check was added before writing each blob to ensure that we would not be overriding an existing blob. Due to S3's weak consistency model, this check was best effort. To make matters worse, however, this resulted in a HEAD request to be done before every PUT, in particular also when PUTTING a new object. The approach taken in #19749 worsened our consistency guarantees for follow-up snapshot actions, as it made it less likely for new files that had been written to be available for reads. This commit therefore removes this extra check. Due to the weak consistency model, this check was a best effort thing anyway, and there's currently no way to prevent accidental overrides on S3.
ywelsch
added a commit
that referenced
this pull request
Jun 11, 2018
There's no need for an extra blobExists() call when writing a blob to the Azure service. Azure provides an option (with stronger consistency guarantees) on the upload method that guarantees that the blob that's uploaded does not already exist. This saves one network roundtrip. Relates to #19749
ywelsch
added a commit
that referenced
this pull request
Jun 11, 2018
There's no need for an extra blobExists() call when writing a blob to the Azure service. Azure provides an option (with stronger consistency guarantees) on the upload method that guarantees that the blob that's uploaded does not already exist. This saves one network roundtrip. Relates to #19749
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enables implementations of the BlobContainer interface to
conform with the requirements of the writeBlob method by
throwing a FileAlreadyExistsException if attempting to write
to a blob that already exists. This change means implementations
of BlobContainer should never overwrite blobs - to overwrite a
blob, it must first be deleted and then can be written again.
Closes #15579