Skip to content

Commit d3ccada

Browse files
dakronedakrone
authored
Add template simulation API for simulating template composition (#56842)
This adds an API for simulating template composition with or without an index template. It looks like: ``` POST /_index_template/_simulate/my-template ``` To simulate a template named `my-template` that already exists, or, to simulate a template that does not already exist: ``` POST /_index_template/_simulate { "index_patterns": ["my-index"] "composed_of": ["ct1", "ct2"], } ``` This is related to #55686, which adds an API to simulate composition based on an index name (hence the `_simulate_index` vs `_simulate`). This commit also adds reference documentation for both simulation APIs. Relates to #53101 Resolves #56390 Resolves #56255
1 parent 0b557e4 commit d3ccada

File tree

12 files changed

+849
-45
lines changed

12 files changed

+849
-45
lines changed

‎client/rest-high-level/src/test/java/org/elasticsearch/client/RestHighLevelClientTests.java‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -803,7 +803,8 @@ public void testApiNamingConventions() throws Exception {
803803
"scripts_painless_execute",
804804
"indices.create_data_stream",
805805
"indices.get_data_stream",
806-
"indices.delete_data_stream"
806+
"indices.delete_data_stream",
807+
"indices.simulate_template"
807808
};
808809
//These API are not required for high-level client feature completeness
809810
String[] notRequiredApi = new String[] {

‎docs/reference/indices/index-templates.asciidoc‎

Lines changed: 130 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ PUT _index_template/template_1
9696
9797
[source,console]
9898
--------------------------------------------------
99-
DELETE _index_template/template_*
99+
DELETE _index_template/*
100100
DELETE _component_template/*
101101
--------------------------------------------------
102102
// TEARDOWN
@@ -291,6 +291,135 @@ PUT /_index_template/template_1
291291
In this case, an index matching `t*` will have three primary shards. If the order of composed
292292
templates were reversed, the index would have two primary shards.
293293

294+
295+
[[simulating-templates]]
296+
===== Simulating template composition
297+
298+
Since templates can be composed not only of multiple component templates, but also the index
299+
template itself, there are two simulation APIs to determine what the resulting index settings will
300+
be.
301+
302+
To simulate the settings that would be applied to a matching index name:
303+
304+
[source,console]
305+
--------------------------------------------------
306+
POST /_index_template/_simulate_index/myindex
307+
--------------------------------------------------
308+
309+
To simulate the settings that would be applied from a particular template:
310+
311+
[source,console]
312+
--------------------------------------------------
313+
POST /_index_template/_simulate/template_1
314+
315+
POST /_index_template/_simulate
316+
{
317+
"index_patterns": ["foo"],
318+
"template": {
319+
"settings": {
320+
"number_of_replicas": 0
321+
}
322+
}
323+
}
324+
--------------------------------------------------
325+
326+
327+
Here's an example demonstrating simulating both an index name and template name:
328+
329+
[source,console]
330+
--------------------------------------------------
331+
PUT /_component_template/ct1 <1>
332+
{
333+
"template": {
334+
"settings": {
335+
"index.number_of_shards": 2
336+
}
337+
}
338+
}
339+
340+
PUT /_component_template/ct2 <2>
341+
{
342+
"template": {
343+
"settings": {
344+
"index.number_of_replicas": 0
345+
},
346+
"mappings": {
347+
"properties": {
348+
"@timestamp": {
349+
"type": "date"
350+
}
351+
}
352+
}
353+
}
354+
}
355+
356+
PUT /_index_template/final-template <3>
357+
{
358+
"index_patterns": ["logdata-*"],
359+
"composed_of": ["ct1", "ct2"],
360+
"priority": 5
361+
}
362+
363+
POST /_index_template/_simulate_index/logdata-2019-02-01 <4>
364+
365+
POST /_index_template/_simulate/final-template <5>
366+
367+
POST /_index_template/_simulate <6>
368+
{
369+
"index_patterns": ["logdata-*"],
370+
"composed_of": ["ct2"],
371+
"priority": 10,
372+
"template": {
373+
"settings": {
374+
"index.number_of_replicas": 1
375+
}
376+
}
377+
}
378+
--------------------------------------------------
379+
<1> Creating a component template (ct1) setting the number of shards to two
380+
<2> Creating another component template (ct2) setting the number of replicas to zero with mappings
381+
<3> Creating an index template called "final" template using ct1 and ct2
382+
<4> Simulate the settings that would be applied for a new index "logdata-2019-02-01"
383+
<5> Simulate the settings composed using the "final-template" index template
384+
<6> Simulate the settings composed using a custom specified template
385+
386+
The output of the simulate API from the last example call looks like:
387+
388+
[source,console-result]
389+
---------------------------------------------------------
390+
{
391+
"template" : {
392+
"settings" : {
393+
"index" : {
394+
"number_of_replicas" : "1" <1>
395+
}
396+
},
397+
"mappings" : {
398+
"properties" : {
399+
"@timestamp" : { <2>
400+
"type" : "date"
401+
}
402+
}
403+
},
404+
"aliases" : { }
405+
},
406+
"overlapping" : [ <3>
407+
{
408+
"name" : "final-template",
409+
"index_patterns" : [
410+
"logdata-*"
411+
]
412+
}
413+
]
414+
}
415+
---------------------------------------------------------
416+
<1> The number of replicas from the simulated template body
417+
<2> The `@timestamp` field inherited from the "ct2" component template
418+
<3> Any overlapping templates that would have matched, but have lower priority
419+
420+
When simulating a template and specifying a template in the body of the request, the simulated
421+
template is not added to the existing templates, it is only used for the simulation.
422+
294423
===== Index template with index aliases
295424

296425
You can include <<indices-aliases,index aliases>> in an index template.

‎rest-api-spec/src/main/resources/rest-api-spec/api/indices.simulate_template.json‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{
2+
"indices.simulate_template":{
3+
"documentation":{
4+
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-templates.html",
5+
"description": "Simulate resolving the given template name or body"
6+
},
7+
"stability":"stable",
8+
"url":{
9+
"paths":[
10+
{
11+
"path":"/_index_template/_simulate",
12+
"methods":[
13+
"POST"
14+
]
15+
},
16+
{
17+
"path":"/_index_template/_simulate/{name}",
18+
"methods":[
19+
"POST"
20+
],
21+
"parts":{
22+
"name":{
23+
"type":"string",
24+
"description":"The name of the index template"
25+
}
26+
}
27+
}
28+
]
29+
},
30+
"params":{
31+
"create":{
32+
"type":"boolean",
33+
"description":"Whether the index template we optionally defined in the body should only be dry-run added if new or can also replace an existing one",
34+
"default":false
35+
},
36+
"cause":{
37+
"type":"string",
38+
"description":"User defined reason for dry-run creating the new template for simulation purposes",
39+
"default":false
40+
},
41+
"master_timeout":{
42+
"type":"time",
43+
"description":"Specify timeout for connection to master"
44+
}
45+
},
46+
"body":{
47+
"description":"New index template definition to be simulated, if no index template name is specified",
48+
"required":false
49+
}
50+
}
51+
}

‎rest-api-spec/src/main/resources/rest-api-spec/test/indices.simulate_index_template/10_basic.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525

2626
- match: {template.settings.index.number_of_shards: "1"}
2727
- match: {template.settings.index.number_of_replicas: "0"}
28-
- match: {template.mappings._doc.properties.field.type: "keyword"}
28+
- match: {template.mappings.properties.field.type: "keyword"}
2929
- match: {overlapping: []}
3030

3131
---
@@ -77,7 +77,7 @@
7777

7878
- match: {template.settings.index.blocks.write: "true"}
7979
- match: {template.settings.index.number_of_replicas: "2"}
80-
- match: {template.mappings._doc.properties.ct_field.type: "keyword"}
80+
- match: {template.mappings.properties.ct_field.type: "keyword"}
8181
- match: {overlapping.0.name: existing_test}
8282
- match: {overlapping.0.index_patterns: ["te*"]}
8383
- length: {template.aliases: 1}
@@ -170,7 +170,7 @@
170170

171171
- match: {template.settings.index.number_of_shards: "1"}
172172
- match: {template.settings.index.number_of_replicas: "0"}
173-
- match: {template.mappings._doc.properties.field.type: "keyword"}
173+
- match: {template.mappings.properties.field.type: "keyword"}
174174
- match: {overlapping.0.name: v1_template}
175175
- match: {overlapping.0.index_patterns: ["t*", "t1*"]}
176176
- match: {overlapping.1.name: v2_template}

‎rest-api-spec/src/main/resources/rest-api-spec/test/indices.simulate_template/10_basic.yml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
---
2+
"Simulate template without a template in the body":
3+
- skip:
4+
version: " - 7.99.99"
5+
reason: "not yet backported"
6+
features: ["default_shards"]
7+
8+
- do:
9+
indices.put_index_template:
10+
name: my-template
11+
body:
12+
index_patterns: other
13+
template:
14+
settings:
15+
number_of_shards: 1
16+
number_of_replicas: 0
17+
mappings:
18+
properties:
19+
field:
20+
type: keyword
21+
22+
- do:
23+
indices.simulate_template:
24+
name: my-template
25+
26+
- match: {template.settings.index.number_of_shards: "1"}
27+
- match: {template.settings.index.number_of_replicas: "0"}
28+
- match: {template.mappings.properties.field.type: "keyword"}
29+
- match: {overlapping: []}
30+
31+
---
32+
"Simulate index template specifying a new template":
33+
- skip:
34+
version: " - 7.99.99"
35+
reason: "not yet backported"
36+
features: ["default_shards"]
37+
38+
- do:
39+
indices.put_index_template:
40+
name: existing_test
41+
body:
42+
index_patterns: te*
43+
priority: 10
44+
template:
45+
settings:
46+
number_of_shards: 1
47+
number_of_replicas: 0
48+
mappings:
49+
properties:
50+
field:
51+
type: keyword
52+
53+
- do:
54+
cluster.put_component_template:
55+
name: ct
56+
body:
57+
template:
58+
settings:
59+
index.number_of_replicas: 2
60+
mappings:
61+
properties:
62+
ct_field:
63+
type: keyword
64+
65+
- do:
66+
indices.simulate_template:
67+
body:
68+
index_patterns: te*
69+
priority: 15
70+
template:
71+
settings:
72+
index.blocks.write: true
73+
aliases:
74+
test_alias: {}
75+
composed_of: ["ct"]
76+
77+
- match: {template.settings.index.blocks.write: "true"}
78+
- match: {template.settings.index.number_of_replicas: "2"}
79+
- match: {template.mappings.properties.ct_field.type: "keyword"}
80+
- match: {overlapping.0.name: existing_test}
81+
- match: {overlapping.0.index_patterns: ["te*"]}
82+
- length: {template.aliases: 1}
83+
- is_true: template.aliases.test_alias
84+
85+
---
86+
"Simulate template matches overlapping V1 and V2 templates":
87+
- skip:
88+
version: " - 7.99.99"
89+
reason: "not yet backported"
90+
features: ["allowed_warnings", "default_shards"]
91+
92+
- do:
93+
indices.put_template:
94+
name: v1_template
95+
body:
96+
index_patterns: [t*, t1*]
97+
settings:
98+
number_of_shards: 5
99+
100+
- do:
101+
allowed_warnings:
102+
- "index template [v2_template] has index patterns [te*] matching patterns from existing older templates [v1_template] with patterns
103+
(v1_template => [t*, t1*]); this template [v2_template] will take precedence during new index creation"
104+
indices.put_index_template:
105+
name: v2_template
106+
body:
107+
index_patterns: te*
108+
priority: 10
109+
template:
110+
settings:
111+
number_of_shards: 10
112+
number_of_replicas: 2
113+
mappings:
114+
properties:
115+
field:
116+
type: text
117+
118+
- do:
119+
allowed_warnings:
120+
- "index template [winning_v2_template] has index patterns [te*] matching patterns from existing older templates [v1_template] with patterns
121+
(v1_template => [t*, t1*]); this template [winning_v2_template] will take precedence during new index creation"
122+
indices.put_index_template:
123+
name: winning_v2_template
124+
body:
125+
index_patterns: te*
126+
priority: 20
127+
template:
128+
settings:
129+
number_of_shards: 1
130+
number_of_replicas: 0
131+
mappings:
132+
properties:
133+
field:
134+
type: keyword
135+
136+
- do:
137+
indices.simulate_template:
138+
name: winning_v2_template
139+
140+
- match: {template.settings.index.number_of_shards: "1"}
141+
- match: {template.settings.index.number_of_replicas: "0"}
142+
- match: {template.mappings.properties.field.type: "keyword"}
143+
- match: {overlapping.0.name: v1_template}
144+
- match: {overlapping.0.index_patterns: ["t*", "t1*"]}
145+
- match: {overlapping.1.name: v2_template}
146+
- match: {overlapping.1.index_patterns: ["te*"]}

0 commit comments

Comments
 (0)