Add configuration to enable Redis Cluster topology refresh

[mp911de]

Redis Cluster can be reconfigured at runtime and Redis clients should to pick up these changes to stay operable. Updating the topology is a specialized topic on its own as Redis Cluster does not push changes to the client but rather clients need to poll topology changes.

Automated (background) topology refresh is currently supported only by Lettuce via ClusterClientOptions and a nested ClusterTopologyRefreshOptions object.
Jedis does not support background topology refreshing.

It would make sense to provide some means to configure cluster topology refresh. The way how Lettuce topology updating is configured (setting ClusterClientOptions via LettuceClientConfigurationBuilder) might contradict with LettuceClientConfigurationBuilderCustomizer.

What do you think?

See also:

• Lettuce via Spring Data Redis/Spring Boot does not refresh cluster topology redis/lettuce#955
• Do cluster slot renewal in a background thread redis/jedis#1347

[wilkinsona]

I don't think that things would necessarily contradict. We could create and set a ClusterClientOptions based on various properties before calling the customisers. They could then further tune things as they do today.

Looking at ClusterClientOptions and ClusterTopologyRefreshOptions, we could add a lot of properties here, and, if we went down that route, I wonder if a user-provided LettuceClientConfigurationBuilderCustomizer would be a better choice.

Are there sensible defaults that could be used such that we could offer a single configuration property that, when set, would cause us to configure the builder with a ClusterClientOptions instance? In other words, does it make sense to do something like this:

if (refreshCluster) {
	builder.clientOptions(ClusterClientOptions.builder().build());
}

[mp911de]

FTR: LettuceClientConfigurationBuilderCustomizer is already in place.

Are there sensible defaults

Not exactly. From the reported requests, each deployment uses different configuration settings. From a configuration perspective and how refresh is used, the most common options are:

• not enable topology refresh at all (that's what happens today)
• enable a light-weight refresh version (ClusterTopologyRefreshOptions.builder().enableAllAdaptiveRefreshTriggers().build())
• periodic refresh (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).build())
• or a combination of both (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).enableAllAdaptiveRefreshTriggers().build()).

In any case, we require a ClusterClientOptions object for ClusterTopologyRefreshOptions.

So how about introducing the following:

• Property to enable refresh (defaults to false, would result in enableAllAdaptiveRefreshTriggers() if enabled).
• Property to enable periodic refresh (no default, would result in enablePeriodicRefresh(Duration) if set and if the refresh property above is enabled).

The background is that adaptive refresh results from using Redis, so Redis responses/connection events are triggers to do a topology refresh. The periodic one requires a scheduler and generates regularly load on all Redis servers even when there was no change.

[wilkinsona]

Thank you, @mp911de. So, how about the following properties:

• spring.redis.cluster.refresh.period (Duration)
• spring.redis.cluster.refresh.all-adaptive-triggers (boolean)

We would only create and set a ClusterClientOptions with a custom ClusterTopologyRefreshOptions if all-adaptive-triggers is true or period in not null.

• Property to enable periodic refresh (no default, would result in enablePeriodicRefresh(Duration) if set and if the refresh property above is enabled).

Why would this require another property to be set? Isn't just configuring the refresh period sufficient to allow people to configure the middle use case in your comment above:

• periodic refresh (ClusterTopologyRefreshOptions.builder().enablePeriodicRefresh(Duration).build())

[mp911de]

Let's rename the second property to spring.redis.cluster.refresh.adaptive as this makes a good story for adaptive and periodic refresh.

Why would this require another property to be set? Isn't just configuration the refresh period sufficient to allow people to configure the middle use case in your comment above:

You're right, maybe it's better to just rely on a single property.

[arganzheng]

mark

[maolujun]

@mp911de Using the jedis client is successful
pom.xml：

[ldwnt]

@mp911de @wilkinsona Do you any plan for including this feature in a new release? I'd really appreciate it if the topology refresh feature could be supported by spring boot, so that my RedisTemplate related code does not need to be re-written~