improve ccr ShardFollowNodeTask computeDelay

[weizijun]

As Randomness.get().nextInt(Math.toIntExact(n+1)) maybe zero, It may cause retry too early. I suggest change from Math.toIntExact(n+1) to nextInt() + 1.

[weizijun]

I also think that, maybe no need Randomness. computeDelay just retry exponential backoff.

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/CCR)

[weizijun]

@ywelsch hi, can you review this pr?

[ywelsch]

@martijnvg can you have a look here?

[martijnvg]

@weizijun Have you run into a situation where if the delay was zero caused issues? Like too many unnecessary retries. I can see why a delay of 0 ms can cause an unnecessary retry, but on the other hand a delay of 0 ms may also be successful, since the delay is just one part of the time it takes to perform a retry.

[weizijun]

@weizijun Have you run into a situation where if the delay was zero caused issues? Like too many unnecessary retries. I can see why a delay of 0 ms can cause an unnecessary retry, but on the other hand a delay of 0 ms may also be successful, since the delay is just one part of the time it takes to perform a retry.

Hi, @martijnvg , some times, The cause of the exception is that the write queue is full (EsRejectedExecutionException). So I would recommend delayed retry.

[dnhatn]

@weizijun We have a new enhancement in 7.9, where the replication component will automatically retry when hitting transient errors. Your case (i.e., EsRejectedExecutionException) is no longer a problem with that change. Hence, I am closing the issue. Please let me know if you think differently. Thank you for your contribution!