Skip to content

Commit 6846666

Browse files
martijnvgmartijnvg
authored
Add ccr follow info api (#37408)
* Add ccr follow info api This api returns all follower indices and per follower index the provided parameters at put follow / resume follow time and whether index following is paused or active. Closes #37127 * iter * [DOCS] Edits the get follower info API * [DOCS] Fixes link to remote cluster * [DOCS] Clarifies descriptions for configured parameters
1 parent 377d96e commit 6846666

File tree

11 files changed

+1048
-19
lines changed

11 files changed

+1048
-19
lines changed

‎docs/reference/ccr/apis/ccr-apis.asciidoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ You can use the following APIs to perform {ccr} operations.
2222
* <<ccr-post-resume-follow,Resume follower>>
2323
* <<ccr-post-unfollow,Convert follower index to a regular index>>
2424
* <<ccr-get-follow-stats,Get stats about follower indices>>
25+
* <<ccr-get-follow-info,Get info about follower indices>>
2526

2627
[float]
2728
[[ccr-api-auto-follow]]
@@ -40,6 +41,7 @@ include::follow/post-pause-follow.asciidoc[]
4041
include::follow/post-resume-follow.asciidoc[]
4142
include::follow/post-unfollow.asciidoc[]
4243
include::follow/get-follow-stats.asciidoc[]
44+
include::follow/get-follow-info.asciidoc[]
4345

4446
// auto-follow
4547
include::auto-follow/put-auto-follow-pattern.asciidoc[]

‎docs/reference/ccr/apis/follow/get-follow-info.asciidoc‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
[role="xpack"]
2+
[testenv="platinum"]
3+
[[ccr-get-follow-info]]
4+
=== Get follower info API
5+
++++
6+
<titleabbrev>Get follower info</titleabbrev>
7+
++++
8+
9+
beta[]
10+
11+
Retrieves information about all follower indices.
12+
13+
==== Description
14+
15+
This API lists the parameters and the status for each follower index.
16+
For example, the results include follower index names, leader index names,
17+
replication options and whether the follower indices are active or paused.
18+
19+
==== Request
20+
21+
//////////////////////////
22+
23+
[source,js]
24+
--------------------------------------------------
25+
PUT /follower_index/_ccr/follow
26+
{
27+
"remote_cluster" : "remote_cluster",
28+
"leader_index" : "leader_index"
29+
}
30+
--------------------------------------------------
31+
// CONSOLE
32+
// TESTSETUP
33+
// TEST[setup:remote_cluster_and_leader_index]
34+
35+
[source,js]
36+
--------------------------------------------------
37+
POST /follower_index/_ccr/pause_follow
38+
--------------------------------------------------
39+
// CONSOLE
40+
// TEARDOWN
41+
42+
//////////////////////////
43+
44+
[source,js]
45+
--------------------------------------------------
46+
GET /<index>/_ccr/info
47+
--------------------------------------------------
48+
// CONSOLE
49+
// TEST[s/<index>/follower_index/]
50+
51+
==== Path Parameters
52+
`index` ::
53+
(string) A comma-delimited list of follower index patterns
54+
55+
==== Results
56+
57+
This API returns the following information:
58+
59+
`follower_indices`::
60+
(array) An array of follower index statistics
61+
62+
The `indices` array consists of objects containing several fields:
63+
64+
`indices[].follower_index`::
65+
(string) The name of the follower index
66+
67+
`indices[].remote_cluster`::
68+
(string) The <<modules-remote-clusters,remote cluster>> that contains the
69+
leader index
70+
71+
`indices[].leader_index`::
72+
(string) The name of the index in the leader cluster that is followed
73+
74+
`indices[].status`::
75+
(string) Whether index following is `active` or `paused`
76+
77+
`indices[].parameters`::
78+
(object) An object that encapsulates {ccr} parameters
79+
80+
The `parameters` contains the following fields:
81+
82+
`indices[].parameters.max_read_request_operation_count`::
83+
(integer) The maximum number of operations to pull per read from the remote
84+
cluster
85+
86+
`indices[].parameters.max_outstanding_read_requests`::
87+
(long) The maximum number of outstanding read requests from the remote cluster
88+
89+
`indices[].parameters.max_read_request_size`::
90+
(<<byte-units,byte value>>) The maximum size in bytes of per read of a batch
91+
of operations pulled from the remote cluster
92+
93+
`indices[].parameters.max_write_request_operation_count`::
94+
(integer) The maximum number of operations per bulk write request executed on
95+
the follower
96+
97+
`indices[].parameters.max_write_request_size`::
98+
(<<byte-units,byte value>>) The maximum total bytes of operations per bulk
99+
write request executed on the follower
100+
101+
`indices[].parameters.max_outstanding_write_requests`::
102+
(integer) The maximum number of outstanding write requests on the follower
103+
104+
`indices[].parameters.max_write_buffer_count`::
105+
(integer) The maximum number of operations that can be queued for writing.
106+
When this limit is reached, reads from the remote cluster are deferred until
107+
the number of queued operations goes below the limit
108+
109+
`indices[].parameters.max_write_buffer_size`::
110+
(<<byte-units,byte value>>) The maximum total bytes of operations that can be
111+
queued for writing. When this limit is reached, reads from the remote cluster
112+
are deferred until the total bytes of queued operations goes below the limit
113+
114+
`indices[].parameters.max_retry_delay`::
115+
(<<time-units,time value>>) The maximum time to wait before retrying an
116+
operation that failed exceptionally. An exponential backoff strategy is
117+
employed when retrying
118+
119+
`indices[].parameters.read_poll_timeout`::
120+
(<<time-units,time value>>) The maximum time to wait for new operations on the
121+
remote cluster when the follower index is synchronized with the leader index.
122+
When the timeout has elapsed, the poll for operations returns to the follower
123+
so that it can update some statistics, then the follower immediately attempts
124+
to read from the leader again
125+
126+
==== Authorization
127+
128+
If the {es} {security-features} are enabled, you must have `monitor` cluster
129+
privileges. For more information, see
130+
{stack-ov}/security-privileges.html[Security privileges].
131+
132+
==== Example
133+
134+
This example retrieves follower info:
135+
136+
[source,js]
137+
--------------------------------------------------
138+
GET /follower_index/_ccr/info
139+
--------------------------------------------------
140+
// CONSOLE
141+
142+
The API returns the following results:
143+
144+
[source,js]
145+
--------------------------------------------------
146+
{
147+
"follower_indices" : [
148+
{
149+
"follower_index" : "follower_index",
150+
"remote_cluster" : "remote_cluster",
151+
"leader_index" : "leader_index",
152+
"status" : "active",
153+
"parameters" : {
154+
"max_read_request_operation_count" : 5120,
155+
"max_read_request_size" : "32mb",
156+
"max_outstanding_read_requests" : 12,
157+
"max_write_request_operation_count" : 5120,
158+
"max_write_request_size" : "9223372036854775807b",
159+
"max_outstanding_write_requests" : 9,
160+
"max_write_buffer_count" : 2147483647,
161+
"max_write_buffer_size" : "512mb",
162+
"max_retry_delay" : "500ms",
163+
"read_poll_timeout" : "1m"
164+
}
165+
}
166+
]
167+
}
168+
--------------------------------------------------
169+
// TESTRESPONSE

‎x-pack/plugin/ccr/qa/rest/src/test/resources/rest-api-spec/test/ccr/follow_info.yml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
"Test info":
3+
- do:
4+
cluster.state: {}
5+
6+
- set: {master_node: master}
7+
8+
- do:
9+
nodes.info: {}
10+
11+
- set: {nodes.$master.transport_address: local_ip}
12+
13+
- do:
14+
cluster.put_settings:
15+
body:
16+
transient:
17+
cluster.remote.local.seeds: $local_ip
18+
flat_settings: true
19+
20+
- match: {transient: {cluster.remote.local.seeds: $local_ip}}
21+
22+
- do:
23+
indices.create:
24+
index: foo
25+
body:
26+
settings:
27+
index:
28+
number_of_shards: 1
29+
number_of_replicas: 0
30+
soft_deletes:
31+
enabled: true
32+
33+
- do:
34+
ccr.follow:
35+
index: bar
36+
body:
37+
remote_cluster: local
38+
leader_index: foo
39+
- is_true: follow_index_created
40+
- is_true: follow_index_shards_acked
41+
- is_true: index_following_started
42+
43+
- do:
44+
ccr.follow_info:
45+
index: bar
46+
- length: {follower_indices: 1}
47+
- match: {follower_indices.0.follower_index: "bar"}
48+
- match: {follower_indices.0.remote_cluster: "local"}
49+
- match: {follower_indices.0.leader_index: "foo"}
50+
- match: {follower_indices.0.status: "active"}
51+
- match: {follower_indices.0.parameters.max_read_request_operation_count: 5120}
52+
- match: {follower_indices.0.parameters.max_read_request_size: "32mb"}
53+
- match: {follower_indices.0.parameters.max_outstanding_read_requests: 12}
54+
- match: {follower_indices.0.parameters.max_write_request_operation_count: 5120}
55+
- match: {follower_indices.0.parameters.max_write_request_size: "9223372036854775807b"}
56+
- match: {follower_indices.0.parameters.max_outstanding_write_requests: 9}
57+
- match: {follower_indices.0.parameters.max_write_buffer_count: 2147483647,}
58+
- match: {follower_indices.0.parameters.max_write_buffer_size: "512mb"}
59+
- match: {follower_indices.0.parameters.max_retry_delay: "500ms"}
60+
- match: {follower_indices.0.parameters.read_poll_timeout: "1m"}
61+
62+
- do:
63+
ccr.pause_follow:
64+
index: bar
65+
- is_true: acknowledged
66+
67+
- do:
68+
ccr.follow_info:
69+
index: bar
70+
- length: {follower_indices: 1}
71+
- match: {follower_indices.0.follower_index: "bar"}
72+
- match: {follower_indices.0.remote_cluster: "local"}
73+
- match: {follower_indices.0.leader_index: "foo"}
74+
- match: {follower_indices.0.status: "paused"}
75+
- is_false: follower_indices.0.parameters

‎x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/Ccr.java‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
import org.elasticsearch.xpack.ccr.action.ShardFollowTasksExecutor;
5151
import org.elasticsearch.xpack.ccr.action.TransportCcrStatsAction;
5252
import org.elasticsearch.xpack.ccr.action.TransportDeleteAutoFollowPatternAction;
53+
import org.elasticsearch.xpack.ccr.action.TransportFollowInfoAction;
5354
import org.elasticsearch.xpack.ccr.action.TransportFollowStatsAction;
5455
import org.elasticsearch.xpack.ccr.action.TransportGetAutoFollowPatternAction;
5556
import org.elasticsearch.xpack.ccr.action.TransportPauseFollowAction;
@@ -69,6 +70,7 @@
6970
import org.elasticsearch.xpack.ccr.repository.CcrRestoreSourceService;
7071
import org.elasticsearch.xpack.ccr.rest.RestCcrStatsAction;
7172
import org.elasticsearch.xpack.ccr.rest.RestDeleteAutoFollowPatternAction;
73+
import org.elasticsearch.xpack.ccr.rest.RestFollowInfoAction;
7274
import org.elasticsearch.xpack.ccr.rest.RestFollowStatsAction;
7375
import org.elasticsearch.xpack.ccr.rest.RestGetAutoFollowPatternAction;
7476
import org.elasticsearch.xpack.ccr.rest.RestPauseFollowAction;
@@ -80,6 +82,7 @@
8082
import org.elasticsearch.xpack.core.ccr.ShardFollowNodeTaskStatus;
8183
import org.elasticsearch.xpack.core.ccr.action.CcrStatsAction;
8284
import org.elasticsearch.xpack.core.ccr.action.DeleteAutoFollowPatternAction;
85+
import org.elasticsearch.xpack.core.ccr.action.FollowInfoAction;
8386
import org.elasticsearch.xpack.core.ccr.action.FollowStatsAction;
8487
import org.elasticsearch.xpack.core.ccr.action.GetAutoFollowPatternAction;
8588
import org.elasticsearch.xpack.core.ccr.action.PauseFollowAction;
@@ -202,6 +205,7 @@ public List<PersistentTasksExecutor<?>> getPersistentTasksExecutor(ClusterServic
202205
// stats action
203206
new ActionHandler<>(FollowStatsAction.INSTANCE, TransportFollowStatsAction.class),
204207
new ActionHandler<>(CcrStatsAction.INSTANCE, TransportCcrStatsAction.class),
208+
new ActionHandler<>(FollowInfoAction.INSTANCE, TransportFollowInfoAction.class),
205209
// follow actions
206210
new ActionHandler<>(PutFollowAction.INSTANCE, TransportPutFollowAction.class),
207211
new ActionHandler<>(ResumeFollowAction.INSTANCE, TransportResumeFollowAction.class),
@@ -225,6 +229,7 @@ public List<RestHandler> getRestHandlers(Settings settings, RestController restC
225229
// stats API
226230
new RestFollowStatsAction(settings, restController),
227231
new RestCcrStatsAction(settings, restController),
232+
new RestFollowInfoAction(settings, restController),
228233
// follow APIs
229234
new RestPutFollowAction(settings, restController),
230235
new RestResumeFollowAction(settings, restController),

0 commit comments

Comments
 (0)