Skip to content

Commit fa7679e

Browse files
s1monws1monw
committed
Add a _freeze / _unfreeze API (#35592)
This commit adds a rest endpoint for freezing and unfreezing an index. Among other cleanups mainly fixing an issue accessing package private APIs from a plugin that got caught by integration tests this change also adds documentation for frozen indices. Note: frozen indices are marked as `beta` and available as a basic feature. Relates to #34352
1 parent 4752552 commit fa7679e

File tree

23 files changed

+796
-118
lines changed

23 files changed

+796
-118
lines changed

‎docs/reference/frozen-indices.asciidoc‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
[role="xpack"]
2+
[testenv="basic"]
3+
[[frozen-indices]]
4+
= Frozen Indices
5+
6+
[partintro]
7+
--
8+
Elasticsearch indices can require a significant amount of memory available in order to be open and searchable. Yet, not all indices need
9+
to be writable at the same time and have different access patterns over time. For example, indices in the time series or logging use cases
10+
are unlikely to be queried once they age out but still need to be kept around for retention policy purposes.
11+
12+
In order to keep indices available and queryable for a longer period but at the same time reduce their hardware requirements they can be transitioned
13+
into a frozen state. Once an index is frozen, all of its transient shard memory (aside from mappings and analyzers)
14+
is moved to persistent storage. This allows for a much higher disk to heap storage ratio on individual nodes. Once an index is
15+
frozen, it is made read-only and drops its transient data structures from memory. These data structures will need to be reloaded on demand (and subsequently dropped) for each search request that targets the frozen index. A search request that hits
16+
one or more frozen shards will be executed on a throttled threadpool that ensures that we never search more than
17+
`N` (`1` by default) searches concurrently (see <<search-throttled>>). This protects nodes from exceeding the available memory due to incoming search requests.
18+
19+
In contrast to ordinary open indices, frozen indices are expected to execute slowly and are not designed for high query load. Parallelism is
20+
gained only on a per-node level and loading data-structures on demand is expected to be one or more orders of a magnitude slower than query
21+
execution on a per shard level. Depending on the data in an index, a frozen index may execute searches in the seconds to minutes range, when the same index in an unfrozen state may execute the same search request in milliseconds.
22+
--
23+
24+
== Best Practices
25+
26+
Since frozen indices provide a much higher disk to heap ratio at the expense of search latency, it is advisable to allocate frozen indices to
27+
dedicated nodes to prevent searches on frozen indices influencing traffic on low latency nodes. There is significant overhead in loading
28+
data structures on demand which can cause page faults and garbage collections, which further slow down query execution.
29+
30+
Since indices that are eligible for freezing are unlikely to change in the future, disk space can be optimized as described in <<tune-for-disk-usage>>.
31+
32+
== Searching a frozen index
33+
34+
Frozen indices are throttled in order to limit memory consumptions per node. The number of concurrently loaded frozen indices per node is
35+
limited by the number of threads in the <<search-throttled>> threadpool, which is `1` by default.
36+
Search requests will not be executed against frozen indices by default, even if a frozen index is named explicitly. This is
37+
to prevent accidental slowdowns by targeting a frozen index by mistake. To include frozen indices a search request must be executed with
38+
the query parameter `ignore_throttled=false`.
39+
40+
[source,js]
41+
--------------------------------------------------
42+
GET /twitter/_search?q=user:kimchy&ignore_throttled=false
43+
--------------------------------------------------
44+
// CONSOLE
45+
// TEST[setup:twitter]
46+
47+
[IMPORTANT]
48+
================================
49+
While frozen indices are slow to search, they can be pre-filtered efficiently. The request parameter `pre_filter_shard_size` specifies
50+
a threshold that, when exceeded, will enforce a round-trip to pre-filter search shards that cannot possibly match.
51+
This filter phase can limit the number of shards significantly. For instance, if a date range filter is applied, then all indices (frozen or unfrozen) that do not contain documents within the date range can be skipped efficiently.
52+
The default value for `pre_filter_shard_size` is `128` but it's recommended to set it to `1` when searching frozen indices. There is no
53+
significant overhead associated with this pre-filter phase.
54+
================================
55+
56+

‎docs/reference/index.asciidoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ include::monitoring/index.asciidoc[]
6565

6666
include::rollup/index.asciidoc[]
6767

68+
include::frozen-indices.asciidoc[]
69+
6870
include::rest-api/index.asciidoc[]
6971

7072
include::commands/index.asciidoc[]

‎docs/reference/indices/apis/freeze.asciidoc‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
[role="xpack"]
2+
[testenv="basic"]
3+
[[freeze-index-api]]
4+
== Freeze Index API
5+
++++
6+
<titleabbrev>Freeze Index</titleabbrev>
7+
++++
8+
9+
Freezes an index.
10+
11+
[float]
12+
=== Request
13+
14+
`POST /<index>/_freeze`
15+
16+
[float]
17+
=== Description
18+
19+
A frozen index has almost no overhead on the cluster (except
20+
for maintaining its metadata in memory), and is blocked for write operations.
21+
See <<frozen-indices>> and <<unfreeze-index-api>>.
22+
23+
[float]
24+
=== Path Parameters
25+
26+
`index` (required)::
27+
(string) Identifier for the index
28+
29+
//=== Query Parameters
30+
31+
//=== Authorization
32+
33+
[float]
34+
=== Examples
35+
36+
The following example freezes and unfreezes an index:
37+
38+
[source,js]
39+
--------------------------------------------------
40+
POST /my_index/_freeze
41+
POST /my_index/_unfreeze
42+
--------------------------------------------------
43+
// CONSOLE
44+
// TEST[s/^/PUT my_index\n/]
45+
46+
[IMPORTANT]
47+
================================
48+
Freezing an index will close the index and reopen it within the same API call. This causes primaries to not be allocated for a short
49+
amount of time and causes the cluster to go red until the primaries are allocated again. This limitation might be removed in the future.
50+
================================

‎docs/reference/indices/apis/unfreeze.asciidoc‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
[role="xpack"]
2+
[testenv="basic"]
3+
[[unfreeze-index-api]]
4+
== Unfreeze Index API
5+
++++
6+
<titleabbrev>Unfreeze Index</titleabbrev>
7+
++++
8+
9+
Unfreezes an index.
10+
11+
[float]
12+
=== Request
13+
14+
`POST /<index>/_unfreeze`
15+
16+
[float]
17+
=== Description
18+
19+
When a frozen index is unfrozen, the index goes through the normal recovery
20+
process and becomes writeable again. See <<frozen-indices>> and <<freeze-index-api>>.
21+
22+
[float]
23+
=== Path Parameters
24+
25+
`index` (required)::
26+
(string) Identifier for the index
27+
28+
29+
//=== Query Parameters
30+
31+
//=== Authorization
32+
33+
[float]
34+
=== Examples
35+
36+
The following example freezes and unfreezes an index:
37+
38+
[source,js]
39+
--------------------------------------------------
40+
POST /my_index/_freeze
41+
POST /my_index/_unfreeze
42+
--------------------------------------------------
43+
// CONSOLE
44+
// TEST[s/^/PUT my_index\n/]
45+
46+
[IMPORTANT]
47+
================================
48+
Freezing an index will close the index and reopen it within the same API call. This causes primaries to not be allocated for a short
49+
amount of time and causes the cluster to go red until the primaries are allocated again. This limitation might be removed in the future.
50+
================================

‎docs/reference/modules/threadpool.asciidoc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,10 @@ There are several thread pools, but the important ones include:
2525
`int((# of available_processors * 3) / 2) + 1`, and initial queue_size of
2626
`1000`.
2727

28+
[[search-throttled]]`search_throttled`::
29+
For count/search/suggest/get operations on `search_throttled indices`. Thread pool type is
30+
`fixed_auto_queue_size` with a size of `1`, and initial queue_size of `100`.
31+
2832
`get`::
2933
For get operations. Thread pool type is `fixed`
3034
with a size of `# of available processors`,

‎docs/reference/rest-api/index.asciidoc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ directly to configure and access {xpack} features.
1010
* <<info-api,Info API>>
1111
* <<ccr-apis,Cross-cluster replication APIs>>
1212
* <<graph-explore-api,Graph Explore API>>
13+
* <<freeze-index-api>>, <<unfreeze-index-api>>
1314
* <<index-lifecycle-management-api,Index lifecycle management APIs>>
1415
* <<licensing-apis,Licensing APIs>>
1516
* <<ml-apis,Machine Learning APIs>>
@@ -23,11 +24,13 @@ directly to configure and access {xpack} features.
2324
include::info.asciidoc[]
2425
include::{es-repo-dir}/ccr/apis/ccr-apis.asciidoc[]
2526
include::{es-repo-dir}/graph/explore.asciidoc[]
27+
include::{es-repo-dir}/indices/apis/freeze.asciidoc[]
2628
include::{es-repo-dir}/ilm/apis/ilm-api.asciidoc[]
2729
include::{es-repo-dir}/licensing/index.asciidoc[]
2830
include::{es-repo-dir}/migration/migration.asciidoc[]
2931
include::{es-repo-dir}/ml/apis/ml-api.asciidoc[]
3032
include::{es-repo-dir}/rollup/rollup-api.asciidoc[]
3133
include::{xes-repo-dir}/rest-api/security.asciidoc[]
34+
include::{es-repo-dir}/indices/apis/unfreeze.asciidoc[]
3235
include::{xes-repo-dir}/rest-api/watcher.asciidoc[]
3336
include::defs.asciidoc[]

‎server/src/main/java/org/elasticsearch/action/admin/indices/close/TransportCloseIndexAction.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ protected void masterOperation(final CloseIndexRequest request, final ClusterSta
109109
.ackTimeout(request.timeout()).masterNodeTimeout(request.masterNodeTimeout())
110110
.indices(concreteIndices);
111111

112-
indexStateService.closeIndex(updateRequest, new ActionListener<ClusterStateUpdateResponse>() {
112+
indexStateService.closeIndices(updateRequest, new ActionListener<ClusterStateUpdateResponse>() {
113113

114114
@Override
115115
public void onResponse(ClusterStateUpdateResponse response) {

‎server/src/main/java/org/elasticsearch/action/admin/indices/open/OpenIndexClusterStateUpdateRequest.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ public class OpenIndexClusterStateUpdateRequest extends IndicesClusterStateUpdat
2828

2929
private ActiveShardCount waitForActiveShards = ActiveShardCount.DEFAULT;
3030

31-
OpenIndexClusterStateUpdateRequest() {
31+
public OpenIndexClusterStateUpdateRequest() {
3232

3333
}
3434

‎server/src/main/java/org/elasticsearch/action/admin/indices/open/OpenIndexResponse.java‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,10 @@ public class OpenIndexResponse extends ShardsAcknowledgedResponse {
4040
declareAcknowledgedAndShardsAcknowledgedFields(PARSER);
4141
}
4242

43-
OpenIndexResponse() {
43+
public OpenIndexResponse() {
4444
}
4545

46-
OpenIndexResponse(boolean acknowledged, boolean shardsAcknowledged) {
46+
public OpenIndexResponse(boolean acknowledged, boolean shardsAcknowledged) {
4747
super(acknowledged, shardsAcknowledged);
4848
}
4949

‎server/src/main/java/org/elasticsearch/cluster/metadata/MetaDataIndexStateService.java‎

Lines changed: 39 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ public MetaDataIndexStateService(ClusterService clusterService, AllocationServic
8484
this.activeShardsObserver = new ActiveShardsObserver(clusterService, threadPool);
8585
}
8686

87-
public void closeIndex(final CloseIndexClusterStateUpdateRequest request, final ActionListener<ClusterStateUpdateResponse> listener) {
87+
public void closeIndices(final CloseIndexClusterStateUpdateRequest request, final ActionListener<ClusterStateUpdateResponse> listener) {
8888
if (request.indices() == null || request.indices().length == 0) {
8989
throw new IllegalArgumentException("Index name is required");
9090
}
@@ -99,46 +99,50 @@ protected ClusterStateUpdateResponse newResponse(boolean acknowledged) {
9999

100100
@Override
101101
public ClusterState execute(ClusterState currentState) {
102-
Set<IndexMetaData> indicesToClose = new HashSet<>();
103-
for (Index index : request.indices()) {
104-
final IndexMetaData indexMetaData = currentState.metaData().getIndexSafe(index);
105-
if (indexMetaData.getState() != IndexMetaData.State.CLOSE) {
106-
indicesToClose.add(indexMetaData);
107-
}
108-
}
102+
return closeIndices(currentState, request.indices(), indicesAsString);
103+
}
104+
});
105+
}
109106

110-
if (indicesToClose.isEmpty()) {
111-
return currentState;
112-
}
107+
public ClusterState closeIndices(ClusterState currentState, final Index[] indices, String indicesAsString) {
108+
Set<IndexMetaData> indicesToClose = new HashSet<>();
109+
for (Index index : indices) {
110+
final IndexMetaData indexMetaData = currentState.metaData().getIndexSafe(index);
111+
if (indexMetaData.getState() != IndexMetaData.State.CLOSE) {
112+
indicesToClose.add(indexMetaData);
113+
}
114+
}
113115

114-
// Check if index closing conflicts with any running restores
115-
RestoreService.checkIndexClosing(currentState, indicesToClose);
116-
// Check if index closing conflicts with any running snapshots
117-
SnapshotsService.checkIndexClosing(currentState, indicesToClose);
118-
logger.info("closing indices [{}]", indicesAsString);
116+
if (indicesToClose.isEmpty()) {
117+
return currentState;
118+
}
119119

120-
MetaData.Builder mdBuilder = MetaData.builder(currentState.metaData());
121-
ClusterBlocks.Builder blocksBuilder = ClusterBlocks.builder()
122-
.blocks(currentState.blocks());
123-
for (IndexMetaData openIndexMetadata : indicesToClose) {
124-
final String indexName = openIndexMetadata.getIndex().getName();
125-
mdBuilder.put(IndexMetaData.builder(openIndexMetadata).state(IndexMetaData.State.CLOSE));
126-
blocksBuilder.addIndexBlock(indexName, INDEX_CLOSED_BLOCK);
127-
}
120+
// Check if index closing conflicts with any running restores
121+
RestoreService.checkIndexClosing(currentState, indicesToClose);
122+
// Check if index closing conflicts with any running snapshots
123+
SnapshotsService.checkIndexClosing(currentState, indicesToClose);
124+
logger.info("closing indices [{}]", indicesAsString);
125+
126+
MetaData.Builder mdBuilder = MetaData.builder(currentState.metaData());
127+
ClusterBlocks.Builder blocksBuilder = ClusterBlocks.builder()
128+
.blocks(currentState.blocks());
129+
for (IndexMetaData openIndexMetadata : indicesToClose) {
130+
final String indexName = openIndexMetadata.getIndex().getName();
131+
mdBuilder.put(IndexMetaData.builder(openIndexMetadata).state(IndexMetaData.State.CLOSE));
132+
blocksBuilder.addIndexBlock(indexName, INDEX_CLOSED_BLOCK);
133+
}
128134

129-
ClusterState updatedState = ClusterState.builder(currentState).metaData(mdBuilder).blocks(blocksBuilder).build();
135+
ClusterState updatedState = ClusterState.builder(currentState).metaData(mdBuilder).blocks(blocksBuilder).build();
130136

131-
RoutingTable.Builder rtBuilder = RoutingTable.builder(currentState.routingTable());
132-
for (IndexMetaData index : indicesToClose) {
133-
rtBuilder.remove(index.getIndex().getName());
134-
}
137+
RoutingTable.Builder rtBuilder = RoutingTable.builder(currentState.routingTable());
138+
for (IndexMetaData index : indicesToClose) {
139+
rtBuilder.remove(index.getIndex().getName());
140+
}
135141

136-
//no explicit wait for other nodes needed as we use AckedClusterStateUpdateTask
137-
return allocationService.reroute(
138-
ClusterState.builder(updatedState).routingTable(rtBuilder.build()).build(),
139-
"indices closed [" + indicesAsString + "]");
140-
}
141-
});
142+
//no explicit wait for other nodes needed as we use AckedClusterStateUpdateTask
143+
return allocationService.reroute(
144+
ClusterState.builder(updatedState).routingTable(rtBuilder.build()).build(),
145+
"indices closed [" + indicesAsString + "]");
142146
}
143147

144148
public void openIndex(final OpenIndexClusterStateUpdateRequest request,

0 commit comments

Comments
 (0)