IndexMetaData: Introduce internal format index setting #25292

spinscale · 2017-06-19T14:05:07Z

This setting is supposed to ease index upgrades as it allows you
to check for a new setting called index.internal.version which
in turn can be configured in the index settings and has to be set
to '6' in order to be valid.

This setting is supposed to ease index upgrades as it allows you to check for a new setting called `index.internal.version` which in turn can be configured in the index settings and has to be set to '6' in order to be valid.

s1monw

left some comments

s1monw · 2017-06-20T13:55:54Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java

+    private static final String INDEX_INTERNAL_FORMAT = "index.internal.format";
+    public static final Setting<Integer> INDEX_INTERNAL_FORMAT_SETTING =
+            Setting.intSetting(INDEX_INTERNAL_FORMAT, 0, Setting.Property.IndexScope);
+    static final int INTERNAL_INDEX_FORMAT_CURRENT = 6;


maybe we make this a bit more generic and call it index.format and INTERNAL_INDEX_FORMAT_CURRENT should not be here. it's up to every user of this setting to define the current?

s1monw · 2017-06-20T13:56:24Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java

    private final Index index;
    private final long version;
    private final long[] primaryTerms;
+    private final int internalIndexFormat;


do we really have to make this a first class citizen or can we just leave it as a setting?

makes more sense as a setting. fixed

s1monw · 2017-06-20T13:56:49Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java

+     * an internal index format description, allowing us to find out if this index is upgraded or needs upgrading
+     */
+    private static final String INDEX_INTERNAL_FORMAT = "index.internal.format";
+    public static final Setting<Integer> INDEX_INTERNAL_FORMAT_SETTING =


also make this setting Setting.Property.Final

oooh, wasnt aware of that. great!

- Make the setting final - Do not expose an own getter, just extract the setting via IndexMetaData.getSettings() - also no need to put this into hashCode()/equals() - Make more generic and rename to `index.format` - Allow the user to choose the correct value and just return that value

spinscale · 2017-06-20T14:35:46Z

all comments made a ton of sense. incorporated.

s1monw

left some comments

s1monw · 2017-06-20T14:57:54Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java

                          DiscoveryNodeFilters requireFilters, DiscoveryNodeFilters initialRecoveryFilters, DiscoveryNodeFilters includeFilters, DiscoveryNodeFilters excludeFilters,
                          Version indexCreatedVersion, Version indexUpgradedVersion,
-                          int routingNumShards, int routingPartitionSize, ActiveShardCount waitForActiveShards) {
+                          int routingNumShards, int routingPartitionSize, ActiveShardCount waitForActiveShards, int internalIndexFormat) {


this is unused?

s1monw · 2017-06-20T14:58:03Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java


            final String uuid = settings.get(SETTING_INDEX_UUID, INDEX_UUID_NA_VALUE);
+
+            int internalIndexFormat = INDEX_FORMAT_SETTING.get(settings);


this is unneeded see above

s1monw · 2017-06-20T14:58:11Z

core/src/main/java/org/elasticsearch/cluster/metadata/IndexMetaData.java

            return new IndexMetaData(new Index(index, uuid), version, primaryTerms, state, numberOfShards, numberOfReplicas, tmpSettings, mappings.build(),
                tmpAliases.build(), customs.build(), filledInSyncAllocationIds.build(), requireFilters, initialRecoveryFilters, includeFilters, excludeFilters,
-                indexCreatedVersion, indexUpgradedVersion, getRoutingNumShards(), routingPartitionSize, waitForActiveShards);
+                indexCreatedVersion, indexUpgradedVersion, getRoutingNumShards(), routingPartitionSize, waitForActiveShards, internalIndexFormat);


this should go away

s1monw

LGTM

This setting is supposed to ease index upgrades as it allows you to check for a new setting called `index.internal.version` which can be used to check before upgrading indices.

* master: (56 commits) Initialize max unsafe auto ID timestamp on shrink Enable a long translog retention policy by default (elastic#25294) Remove `index.mapping.single_type=false` from core/tests (elastic#25331) test: single type defaults to true since alpha1 and not alpha3 Get short path name for native controllers Live primary-replica resync (no rollback) (elastic#24841) Upgrade to lucene-7.0.0-snapshot-ad2cb77. (elastic#25349) percolator: Deprecate `document_type` parameter. [DOCS] Fixed typo. [rest-api-spec/indices.refresh] Remove old params Remove redundant and broken MD5 checksum from repository-s3 (elastic#25270) Initialize sequence numbers on a shrunken index Port most snapshot/restore static bwc tests to qa:full-cluster-restart (elastic#25296) Javadoc: ThreadPool doesn't reject while shutdown (elastic#23678) test: verify `size_to_upgrade_in_bytes` in assertBusy(...) Docs: Removed duplicated line in mapping docs Add backward compatibility indices for 5.4.2 Update MockTransportService to the age of Transport.Connection (elastic#25320) Add version v5.4.2 after release IndexMetaData: Add internal format index setting (elastic#25292) ...

IndexMetaData: Introduce internal format index setting

5991ae6

This setting is supposed to ease index upgrades as it allows you to check for a new setting called `index.internal.version` which in turn can be configured in the index settings and has to be set to '6' in order to be valid.

spinscale added review >upgrade labels Jun 19, 2017

spinscale requested a review from s1monw June 19, 2017 14:05

spinscale added v5.6.0 v6.0.0 labels Jun 19, 2017

Merge branch 'master' into 1706-add-internal-index-format

8f9b2e4

s1monw suggested changes Jun 20, 2017

View reviewed changes

spinscale added 2 commits June 20, 2017 16:22

Merge branch 'master' into 1706-add-internal-index-format

b8d51fa

s1monw suggested changes Jun 20, 2017

View reviewed changes

removed unused code

bbffd94

s1monw approved these changes Jun 20, 2017

View reviewed changes

spinscale merged commit 6842398 into elastic:master Jun 21, 2017

clintongormley added v6.0.0-beta1 :Core/Infra/Settings Settings infrastructure and APIs >enhancement and removed v6.0.0 >upgrade labels Jul 25, 2017


		final String uuid = settings.get(SETTING_INDEX_UUID, INDEX_UUID_NA_VALUE);

		int internalIndexFormat = INDEX_FORMAT_SETTING.get(settings);

IndexMetaData: Introduce internal format index setting #25292

IndexMetaData: Introduce internal format index setting #25292

Uh oh!

Conversation

spinscale commented Jun 19, 2017

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spinscale commented Jun 20, 2017

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s1monw left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants