Skip to content

Commit 6cf85f9

Browse files
authored
Rolledback compactor blocks sharding (#2628)
* Rolledback compactor blocks sharding Signed-off-by: Marco Pracucci <[email protected]> * Fixes Signed-off-by: Marco Pracucci <[email protected]>
1 parent 9993aac commit 6cf85f9

File tree

10 files changed

+133
-269
lines changed

10 files changed

+133
-269
lines changed

CHANGELOG.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@
2929
* `cortex_storegateway_blocks_last_successful_sync_timestamp_seconds`
3030
* [ENHANCEMENT] Experimental TSDB: added the flag `-experimental.tsdb.wal-compression-enabled` to allow to enable TSDB WAL compression. #2585
3131
* [ENHANCEMENT] Experimental TSDB: Querier and store-gateway components can now use so-called "caching bucket", which can currently cache fetched chunks into shared memcached server. #2572
32-
* [ENHANCEMENT] Experimental TSDB: added per-tenant blocks sharding support in the compactor, in order to parallelize blocks compaction for a single tenant in a single node. Sharding can be enabled via `-compactor.per-tenant-num-shards` while the parallelization can be controlled with `-compactor.per-tenant-shards-concurrency`. #2599
3332
* [ENHANCEMENT] Ruler: Automatically remove unhealthy rulers from the ring. #2587
3433
* [ENHANCEMENT] `query-tee` supports `/metadata`, `/alerts`, and `/rules` #2600
3534
* [ENHANCEMENT] Thanos and Prometheus upgraded to [806479182a6b](https://github.com/thanos-io/thanos/commit/806479182a6b) and [cd73b3d33e06](https://github.com/prometheus/prometheus/commit/cd73b3d33e06) respectively. #2604

docs/configuration/config-file-reference.md

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3011,7 +3011,7 @@ The `compactor_config` configures the compactor for the experimental blocks stor
30113011
# Malformed blocks older than the maximum of consistency-delay and 48h0m0s will
30123012
# be removed.
30133013
# CLI flag: -compactor.consistency-delay
3014-
[consistency_delay: <duration> | default = 30m]
3014+
[consistency_delay: <duration> | default = 0s]
30153015
30163016
# Data directory in which to cache blocks and process compactions
30173017
# CLI flag: -compactor.data-dir
@@ -3026,6 +3026,10 @@ The `compactor_config` configures the compactor for the experimental blocks stor
30263026
# CLI flag: -compactor.compaction-retries
30273027
[compaction_retries: <int> | default = 3]
30283028
3029+
# Max number of concurrent compactions running.
3030+
# CLI flag: -compactor.compaction-concurrency
3031+
[compaction_concurrency: <int> | default = 1]
3032+
30293033
# Time before a block marked for deletion is deleted from bucket. If not 0,
30303034
# blocks will be marked for deletion and compactor component will delete blocks
30313035
# marked for deletion from the bucket. If delete-delay is 0, blocks will be
@@ -3086,15 +3090,6 @@ sharding_ring:
30863090
# the ring.
30873091
# CLI flag: -compactor.ring.heartbeat-timeout
30883092
[heartbeat_timeout: <duration> | default = 1m]
3089-
3090-
# Number of shards a single tenant blocks should be grouped into (0 or 1 means
3091-
# per-tenant blocks sharding is disabled).
3092-
# CLI flag: -compactor.per-tenant-num-shards
3093-
[per_tenant_num_shards: <int> | default = 1]
3094-
3095-
# Number of concurrent shards compacted for a single tenant.
3096-
# CLI flag: -compactor.per-tenant-shards-concurrency
3097-
[per_tenant_shards_concurrency: <int> | default = 1]
30983093
```
30993094

31003095
### `store_gateway_config`

docs/operations/blocks-storage.md

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -620,7 +620,7 @@ compactor:
620620
# Malformed blocks older than the maximum of consistency-delay and 48h0m0s
621621
# will be removed.
622622
# CLI flag: -compactor.consistency-delay
623-
[consistency_delay: <duration> | default = 30m]
623+
[consistency_delay: <duration> | default = 0s]
624624
625625
# Data directory in which to cache blocks and process compactions
626626
# CLI flag: -compactor.data-dir
@@ -635,6 +635,10 @@ compactor:
635635
# CLI flag: -compactor.compaction-retries
636636
[compaction_retries: <int> | default = 3]
637637
638+
# Max number of concurrent compactions running.
639+
# CLI flag: -compactor.compaction-concurrency
640+
[compaction_concurrency: <int> | default = 1]
641+
638642
# Time before a block marked for deletion is deleted from bucket. If not 0,
639643
# blocks will be marked for deletion and compactor component will delete
640644
# blocks marked for deletion from the bucket. If delete-delay is 0, blocks
@@ -696,15 +700,6 @@ compactor:
696700
# within the ring.
697701
# CLI flag: -compactor.ring.heartbeat-timeout
698702
[heartbeat_timeout: <duration> | default = 1m]
699-
700-
# Number of shards a single tenant blocks should be grouped into (0 or 1 means
701-
# per-tenant blocks sharding is disabled).
702-
# CLI flag: -compactor.per-tenant-num-shards
703-
[per_tenant_num_shards: <int> | default = 1]
704-
705-
# Number of concurrent shards compacted for a single tenant.
706-
# CLI flag: -compactor.per-tenant-shards-concurrency
707-
[per_tenant_shards_concurrency: <int> | default = 1]
708703
```
709704

710705
## Known issues

pkg/compactor/blocks_sharding_filter.go

Lines changed: 0 additions & 66 deletions
This file was deleted.

pkg/compactor/blocks_sharding_filter_test.go

Lines changed: 0 additions & 152 deletions
This file was deleted.

0 commit comments

Comments
 (0)