Add to HealthMetadata information about ShardLimits #94116

HiDAl · 2023-02-24T14:17:45Z

create all the machinery to have the shard limits information from the master node. Later, these values will be used by the new shard limits health indicator.

adds a new ShardLimitsMetadata object to be filled up by the master node and used by the designed health-node
wire up the listeners for changes of shard limits-related settings

issue #91119

elasticsearchmachine · 2023-02-24T14:18:09Z

Pinging @elastic/es-data-management (Team:Data Management)

elasticsearchmachine · 2023-02-24T14:20:34Z

Hi @HiDAl, I've created a changelog YAML for you.

HiDAl · 2023-02-24T14:52:20Z

@elasticsearchmachine run elasticsearch-ci/bwc

HiDAl · 2023-02-24T16:35:48Z

BWC tests are not passing due to this:

 Caused by: org.elasticsearch.transport.RemoteTransportException: [v8.6.3-0][127.0.0.1:33721][internal:cluster/coordination/join/validate]
16:00:18 »  Caused by: java.io.EOFException
16:00:18 »  	at org.elasticsearch.common.io.stream.InputStreamStreamInput.readByte(InputStreamStreamInput.java:47)
16:00:18 »  	at org.elasticsearch.common.io.stream.StreamInput.readInt(StreamInput.java:187)
16:00:18 »  	at org.elasticsearch.common.io.stream.FilterStreamInput.readInt(FilterStreamInput.java:50)
16:00:18 »  	at org.elasticsearch.health.metadata.HealthMetadata$ShardLimits.readFrom(HealthMetadata.java:128)
16:00:18 »  	at org.elasticsearch.health.metadata.HealthMetadata.<init>(HealthMetadata.java:51)
16:00:18 »  	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:56)
16:00:18 »  	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:32)
16:00:18 »  	at org.elasticsearch.cluster.ClusterState.readFrom(ClusterState.java:841)
16:00:18 »  	at org.elasticsearch.cluster.coordination.ValidateJoinRequest.readCompressed(ValidateJoinRequest.java:59)
16:00:18 »  	at org.elasticsearch.cluster.coordination.ValidateJoinRequest.lambda$new$0(ValidateJoinRequest.java:36)
16:00:18 »  	at org.elasticsearch.cluster.coordination.ValidateJoinRequest.getOrReadState(ValidateJoinRequest.java:76)
16:00:18 »  	at org.elasticsearch.cluster.coordination.JoinValidationService.lambda$new$2(JoinValidationService.java:121)
16:00:18 »  	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:71)
16:00:18 »  	at org.elasticsearch.transport.InboundHandler$1.doRun(InboundHandler.java:283)
16:00:18 »  	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:958)
16:00:18 »  	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
16:00:18 »  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
16:00:18 »  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
16:00:18 »  	at java.lang.Thread.run(Thread.java:833)

converting to a draft in order of fix the issue first

HiDAl · 2023-02-27T13:14:05Z

@elasticsearchmachine run elasticsearch-ci/bwc

andreidan

Thanks Pablo, this generally looks great.

I left a few suggestions

server/src/internalClusterTest/java/org/elasticsearch/health/HealthMetadataServiceIT.java

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadata.java

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadataService.java

gmarouli

Thank for working on this, it's nice to see the HealthMetadata being extended :). I left some minor comments and I second Andrei's comments :).

server/src/internalClusterTest/java/org/elasticsearch/health/HealthMetadataServiceIT.java

server/src/test/java/org/elasticsearch/health/metadata/HealthMetadataSerializationTests.java

andreidan · 2023-03-01T13:19:07Z

elasticsearch-ci/bwc failure seems transient. Restarting

andreidan · 2023-03-01T13:19:14Z

@elasticmachine run elasticsearch-ci/bwc

andreidan

LGTM, thanks for iterating on this Pablo

HiDAl · 2023-03-02T10:19:14Z

@elasticmachine update branch

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadata.java

fix typo on method/variable names

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadata.java

ShardLimits validation shouldn't run as part of the construction of the object, otherwise it will block the serialization path.

when serializing the value to the wire, we ensure that it's writable/readable using Optional methods. add proper tests to check the null case

HiDAl · 2023-03-02T12:39:02Z

executed the serialization test 1000 times, no errors:

➜  elasticsearch git:(new-shard-limit-metadata) ✗ ./gradlew ':server:test' --tests "org.elasticsearch.health.metadata.HealthMetadataSerializationTests" -Dtests.iters=1000
=======================================
...
=======================================

> Task :server:test
[...]
BUILD SUCCESSFUL in 18s
61 actionable tasks: 2 executed, 59 up-to-date

Publishing build scan...
https://gradle-enterprise.elastic.co/s/endahupjxcp4y

HiDAl · 2023-03-02T13:31:52Z

@elasticmachine run elasticsearch-ci/part-2

HiDAl added >feature Team:Data Management Meta label for data/management team :Data Management/Health labels Feb 24, 2023

HiDAl requested review from andreidan and gmarouli February 24, 2023 14:17

elasticsearchmachine added the v8.8.0 label Feb 24, 2023

HiDAl marked this pull request as draft February 24, 2023 16:36

HiDAl force-pushed the new-shard-limit-metadata branch from 4c1e61e to a07e5be Compare February 24, 2023 18:06

HiDAl marked this pull request as ready for review February 28, 2023 08:12

andreidan reviewed Feb 28, 2023

View reviewed changes

gmarouli reviewed Feb 28, 2023

View reviewed changes

HiDAl mentioned this pull request Feb 28, 2023

Remove unneeded argument from ShardLimitValidator #94193

Merged

Pablo Alcantar Morales and others added 9 commits March 1, 2023 11:38

new record object to store shardLimits information

9289193

add to HealthMetadata the new ShardLimits info

a83f335

add listeners for shard limits setting changes

a00dbf9

add tests for shard limits changes

1b46288

Update docs/changelog/94116.yaml

25e0aca

mark the minimum version since when the metadata is available

db4ae00

fix versioning of shardLimits read/write

a07a2c5

spotless

d8e255c

tackle PR comments

68fb6e2

HiDAl force-pushed the new-shard-limit-metadata branch from 9468d2f to 68fb6e2 Compare March 1, 2023 10:38

HiDAl requested review from andreidan and gmarouli March 1, 2023 10:41

andreidan approved these changes Mar 1, 2023

View reviewed changes

Merge branch 'main' into new-shard-limit-metadata

aeaa2f6

andreidan reviewed Mar 2, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadata.java Outdated Show resolved Hide resolved

remove unused method

25bc07f

fix typo on method/variable names

andreidan reviewed Mar 2, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/health/metadata/HealthMetadata.java Outdated Show resolved Hide resolved

Pablo Alcantar Morales added 2 commits March 2, 2023 12:52

move the validation to the Builders

29e63af

ShardLimits validation shouldn't run as part of the construction of the object, otherwise it will block the serialization path.

ensure that shardLimitsMetadata can be null

d249cd5

when serializing the value to the wire, we ensure that it's writable/readable using Optional methods. add proper tests to check the null case

remove unneeded tests

54f71f3

HiDAl mentioned this pull request Mar 2, 2023

[HealthAPI] Warn when max shards per node setting is reached #91119

Closed

7 tasks

HiDAl merged commit d10a976 into elastic:main Mar 2, 2023

HiDAl deleted the new-shard-limit-metadata branch March 2, 2023 14:16

Add to HealthMetadata information about ShardLimits #94116

Add to HealthMetadata information about ShardLimits #94116

Uh oh!

Conversation

HiDAl commented Feb 24, 2023

Uh oh!

elasticsearchmachine commented Feb 24, 2023

Uh oh!

elasticsearchmachine commented Feb 24, 2023

Uh oh!

HiDAl commented Feb 24, 2023

Uh oh!

HiDAl commented Feb 24, 2023

Uh oh!

HiDAl commented Feb 27, 2023

Uh oh!

andreidan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gmarouli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andreidan commented Mar 1, 2023

Uh oh!

andreidan commented Mar 1, 2023

Uh oh!

andreidan left a comment

Choose a reason for hiding this comment

Uh oh!

HiDAl commented Mar 2, 2023

Uh oh!

Uh oh!

Uh oh!

HiDAl commented Mar 2, 2023

Uh oh!

HiDAl commented Mar 2, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants