Skip to content

Commit 515a233

Browse files
authored
Do not check for S3 blob to exist before writing (#31128)
In #19749 an extra check was added before writing each blob to ensure that we would not be overriding an existing blob. Due to S3's weak consistency model, this check was best effort. To make matters worse, however, this resulted in a HEAD request to be done before every PUT, in particular also when PUTTING a new object. The approach taken in #19749 worsened our consistency guarantees for follow-up snapshot actions, as it made it less likely for new files that had been written to be available for reads. This commit therefore removes this extra check. Due to the weak consistency model, this check was a best effort thing anyway, and there's currently no way to prevent accidental overrides on S3.
1 parent 8aa5888 commit 515a233

File tree

2 files changed

+5
-4
lines changed

2 files changed

+5
-4
lines changed

plugins/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3BlobContainer.java

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -96,10 +96,6 @@ public InputStream readBlob(String blobName) throws IOException {
9696

9797
@Override
9898
public void writeBlob(String blobName, InputStream inputStream, long blobSize) throws IOException {
99-
if (blobExists(blobName)) {
100-
throw new FileAlreadyExistsException("Blob [" + blobName + "] already exists, cannot overwrite");
101-
}
102-
10399
SocketAccess.doPrivilegedIOException(() -> {
104100
if (blobSize <= blobStore.bufferSizeInBytes()) {
105101
executeSingleUpload(blobStore, buildKey(blobName), inputStream, blobSize);

plugins/repository-s3/src/test/java/org/elasticsearch/repositories/s3/S3BlobStoreContainerTests.java

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,11 @@ protected BlobStore newBlobStore() {
6464
return randomMockS3BlobStore();
6565
}
6666

67+
@Override
68+
public void testVerifyOverwriteFails() {
69+
assumeFalse("not implemented because of S3's weak consistency model", true);
70+
}
71+
6772
public void testExecuteSingleUploadBlobSizeTooLarge() {
6873
final long blobSize = ByteSizeUnit.GB.toBytes(randomIntBetween(6, 10));
6974
final S3BlobStore blobStore = mock(S3BlobStore.class);

0 commit comments

Comments
 (0)