Fix remove of azure files #117
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Probably when we updated Azure SDK, we introduced a regression.
Actually, we are not able to remove files anymore.
For example, if you register a new azure repository, the snapshot service tries to create a temp file and then remove it.
Removing does not work and you can see in logs:
```
[2016-05-18 11:03:24,914][WARN ][org.elasticsearch.cloud.azure.blobstore] [azure] can not remove [tests-ilmRPJ8URU-sh18yj38O6g/] in container {elasticsearch-snapshots}: The specified blob does not exist.
```
This fix deals with that. It now list all the files in a flatten mode, remove in the full URL the server and the container name.
As an example, when you are removing a blob which full name is `https://dpi24329.blob.core.windows.net/elasticsearch-snapshots/bar/test` you need to actually call Azure SDK with `bar/test` as the path, `elasticsearch-snapshots` is the container.
Related to elastic/elasticsearch#16472.
Related to elastic/elasticsearch#18436.
Backport of elastic/elasticsearch#18451 for 1.7 series
Contributor
Author
|
@imotov Same here for 1.7 series. Could you look at it please? |
|
LGTM |
Contributor
Author
|
If anyone wants to test a SNAPSHOT version before we release: bin/plugin --remove cloud-azure
bin/plugin --url https://oss.sonatype.org/content/repositories/snapshots/org/elasticsearch/elasticsearch-cloud-azure/2.8.4-SNAPSHOT/elasticsearch-cloud-azure-2.8.4-20160527.150319-1.zip --install cloud-azureThen create a new repo, snapshot an index. Check on Azure that files are here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Probably when we updated Azure SDK, we introduced a regression.
Actually, we are not able to remove files anymore.
For example, if you register a new azure repository, the snapshot service tries to create a temp file and then remove it.
Removing does not work and you can see in logs:
This fix deals with that. It now list all the files in a flatten mode, remove in the full URL the server and the container name.
As an example, when you are removing a blob which full name is
https://dpi24329.blob.core.windows.net/elasticsearch-snapshots/bar/testyou need to actually call Azure SDK withbar/testas the path,elasticsearch-snapshotsis the container.Related to elastic/elasticsearch#16472.
Related to elastic/elasticsearch#18436.
Backport of elastic/elasticsearch#18451 for 1.7 series