-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Description
Elasticsearch version: upgrading from anything <5.6.3
Description of the problem including expected versus actual behavior:
With versions earlier than 5.6.3 , Elasticsearch could end up with an alias with the same name as an index. For details, refer to #26804
There have been reports of users upgrading Elasticsearch while this condition is in place, which results in the service failing to start after the newer package is installed.
Say there is an alias .name pointing to the index .name-alias , and an index named after .name
Upgrading Elasticsearch to a later version will result to the following error and will abort during boot:
[WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hostname] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: index and alias names need to be unique, but the following duplicates were found [.name (alias of [.name-alias/id])]
The master node logs warn about the situation but the issue becomes a major blocker during an upgrade:
[WARN ][o.e.g.LocalAllocateDangledIndices] [esmaster] ignoring dangled index [[.name/id]] on node [{datanode}{id}] due to an existing alias with the same name
One way to get out of this situation, when a data node hits the bootstrap error and shuts down is to locate the physical directory of the problem index on the data path and remove it (purge the index). Not ideal.
This can also manifest when the Reindex Helper in the Kibana Upgrade Assistant (X-Pack 5.6) creates a .security and a .kibana aliases pointing to .security-6 and .kibana-6.
I am unsure which version introduced the abort behavior while booting.
To avoid this, there should be step in docs for the user to verify that there is no such problematic index-alias naming overlap when upgrading from <5.6.3 to a new version.
Especially for the 5.6 to 6.x upgrade it would be good to recommend upgrading only from the latest minor 5.6.x version to 6+