Data tier ergonomics #70618
Description
The formalization of data tiers (#60994) enables us to ship with defaults that are better suited for the expected usage pattern.
Content indices
If we can expect content indices to be more actively searched than indexed, we could make the following changes, which would trade indexing speed for search speed or
- Number of segments per tier set to 5 (default: 10)?
- Maximum percentage of deletes set to 20% (default: 33%)?
- Enable merge-on-refresh (default: disabled)?
- Refresh interval set to
1s(default: disabled on inactive shards,1son active shards)?
Hot indices
Our defaults tend to be geared towards hot data, is there any setting that could have a more sensible value on hot data?
Warm indices
Warm, cold and frozen indices are expected to be allocated to dense storage. Memory needs to be wisely used.
- Disable query cache (Disable query cache for density-optimized use-cases? #59846)
- Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)
Cold indices
- Disable query cache (Disable query cache for density-optimized use-cases? #59846)
- Automatically leverage FrozenEngine (Should we automatically use the FrozenEngine on cold and frozen tiers? #71002)
- Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)
Frozen indices
- Disable query cache (Disable query cache for density-optimized use-cases? #59846)
- Automatically leverage FrozenEngine (Should we automatically use the FrozenEngine on cold and frozen tiers? #71002)
- Group shard search requests that hit the same shard together (Group together search requests that hit the same shard in the search queue #71962)