From c0d381bd577f7db1fb6eadab00aa44ba902c04f3 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 31 Dec 2025 11:05:18 +0800 Subject: [PATCH 1/2] v8.5.4: tikv: update auto compaction config --- garbage-collection-configuration.md | 4 ++-- tikv-configuration-file.md | 32 ++++++++++++++--------------- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/garbage-collection-configuration.md b/garbage-collection-configuration.md index f06b20d5c36d0..fe4a6f9e4cf38 100644 --- a/garbage-collection-configuration.md +++ b/garbage-collection-configuration.md @@ -116,7 +116,7 @@ show config where type = 'tikv' and name like '%enable-compaction-filter%'; > When using the Compaction Filter mechanism, GC progress might be delayed, which can affect TiKV scan performance. If your workload contains a large number of coprocessor requests and you observe in the [**TiKV-Details > Coprocessor Detail**](/grafana-tikv-dashboard.md#coprocessor-detail) panel that the `next()` or `prev()` call count in **Total Ops Details** significantly exceeds three times the `processed_keys` calls, you can take the following actions: > > - For TiDB versions before v7.1.3, it is recommended to disable Compaction Filter to speed up GC. -> - For TiDB versions from v7.1.3 to v7.5.6, TiDB automatically triggers compaction based on the number of redundant versions in each Region [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and the percentage of redundant versions [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) to improve Compaction Filter GC performance. In this case, adjust these configuration items instead of disabling Compaction Filter. -> - Starting from v7.5.7, [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) are deprecated. TiDB now automatically triggers compaction based on [`gc.auto-compaction.redundant-rows-threshold`](/tikv-configuration-file.md#redundant-rows-threshold-new-in-v757-and-v900) and [`gc.auto-compaction.redundant-rows-percent-threshold`](/tikv-configuration-file.md#redundant-rows-percent-threshold-new-in-v757-and-v900). In this case, adjust these configuration items instead of disabling Compaction Filter. +> - For TiDB versions from v7.1.3 to v7.5.6 and v7.6.0 to v8.5.3, TiDB automatically triggers compaction based on the number of redundant versions in each Region [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and the percentage of redundant versions [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) to improve Compaction Filter GC performance. In this case, adjust these configuration items instead of disabling Compaction Filter. +> - Starting from v7.5.7 and v8.5.4, [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) are deprecated. TiDB now automatically triggers compaction based on [`gc.auto-compaction.redundant-rows-threshold`](/tikv-configuration-file.md#redundant-rows-threshold-new-in-v757-v854-and-v900) and [`gc.auto-compaction.redundant-rows-percent-threshold`](/tikv-configuration-file.md#redundant-rows-percent-threshold-new-in-v757-v854-and-v900). In this case, adjust these configuration items instead of disabling Compaction Filter. diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 84a578de8c166..6a2ae0daf34e2 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -819,7 +819,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.check-interval`](#check-interval-new-in-v757-and-v900). +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.check-interval`](#check-interval-new-in-v757-v854-and-v900). + The time interval at which to check whether it is necessary to manually trigger RocksDB compaction. `0` means that this feature is disabled. + Default value: `"5m"` @@ -829,7 +829,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated. +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated. + The number of Regions checked at one time for each round of manual compaction + Default value: @@ -842,7 +842,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.tombstone-num-threshold`](#tombstone-num-threshold-new-in-v757-and-v900). +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.tombstone-num-threshold`](#tombstone-num-threshold-new-in-v757-v854-and-v900). + The number of tombstones required to trigger RocksDB compaction + Default value: `10000` @@ -852,7 +852,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.tombstone-percent-threshold`](#tombstone-percent-threshold-new-in-v757-and-v900). +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.tombstone-percent-threshold`](#tombstone-percent-threshold-new-in-v757-v854-and-v900). + The proportion of tombstone required to trigger RocksDB compaction + Default value: `30` @@ -863,7 +863,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.redundant-rows-threshold`](#redundant-rows-threshold-new-in-v757-and-v900). +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.redundant-rows-threshold`](#redundant-rows-threshold-new-in-v757-v854-and-v900). + The number of redundant MVCC rows required to trigger RocksDB compaction. + Default value: `50000` @@ -873,7 +873,7 @@ Configuration items related to Raftstore. > **Warning:** > -> Starting from v7.5.7 and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.redundant-rows-percent-threshold`](#redundant-rows-percent-threshold-new-in-v757-and-v900). +> Starting from v7.5.7, v8.5.4, and v9.0.0, this configuration item is deprecated and replaced by [`gc.auto-compaction.redundant-rows-percent-threshold`](#redundant-rows-percent-threshold-new-in-v757-v854-and-v900). + The percentage of redundant MVCC rows required to trigger RocksDB compaction. + Default value: `20` @@ -2283,42 +2283,42 @@ Configuration items related to TiDB Lightning import and BR restore. Configures the behavior of TiKV automatic compaction. -### `check-interval` New in v7.5.7 and v9.0.0 +### `check-interval` New in v7.5.7, v8.5.4, and v9.0.0 + The interval at which TiKV checks whether to trigger automatic compaction. Within this interval, Regions that meet the automatic compaction conditions are processed based on priority. When the interval elapses, TiKV rescans Region information and recalculates priorities. + Default value: `"300s"` -### `tombstone-num-threshold` New in v7.5.7 and v9.0.0 +### `tombstone-num-threshold` New in v7.5.7, v8.5.4, and v9.0.0 -+ The number of RocksDB tombstones required to trigger TiKV automatic compaction. When the number of tombstones reaches this threshold, or when the percentage of tombstones reaches [`tombstone-percent-threshold`](#tombstone-percent-threshold-new-in-v757-and-v900), TiKV triggers automatic compaction. ++ The number of RocksDB tombstones required to trigger TiKV automatic compaction. When the number of tombstones reaches this threshold, or when the percentage of tombstones reaches [`tombstone-percent-threshold`](#tombstone-percent-threshold-new-in-v757-v854-and-v900), TiKV triggers automatic compaction. + This configuration item takes effect only when [Compaction Filter](/garbage-collection-configuration.md) is disabled. + Default value: `10000` + Minimum value: `0` -### `tombstone-percent-threshold` New in v7.5.7 and v9.0.0 +### `tombstone-percent-threshold` New in v7.5.7, v8.5.4, and v9.0.0 -+ The percentage of RocksDB tombstones required to trigger TiKV automatic compaction. When the percentage of tombstones reaches this threshold, or when the number of tombstones reaches [`tombstone-num-threshold`](#tombstone-num-threshold-new-in-v757-and-v900), TiKV triggers automatic compaction. ++ The percentage of RocksDB tombstones required to trigger TiKV automatic compaction. When the percentage of tombstones reaches this threshold, or when the number of tombstones reaches [`tombstone-num-threshold`](#tombstone-num-threshold-new-in-v757-v854-and-v900), TiKV triggers automatic compaction. + This configuration item takes effect only when [Compaction Filter](/garbage-collection-configuration.md) is disabled. + Default value: `30` + Minimum value: `0` + Maximum value: `100` -### `redundant-rows-threshold` New in v7.5.7 and v9.0.0 +### `redundant-rows-threshold` New in v7.5.7, v8.5.4, and v9.0.0 -+ The number of redundant MVCC rows required to trigger TiKV automatic compaction. Redundant rows include RocksDB tombstones, TiKV stale versions, and TiKV deletion tombstones. When the number of redundant MVCC rows reaches this threshold, or when the percentage of these rows reaches [`redundant-rows-percent-threshold`](#redundant-rows-percent-threshold-new-in-v757-and-v900), TiKV triggers automatic compaction. ++ The number of redundant MVCC rows required to trigger TiKV automatic compaction. Redundant rows include RocksDB tombstones, TiKV stale versions, and TiKV deletion tombstones. When the number of redundant MVCC rows reaches this threshold, or when the percentage of these rows reaches [`redundant-rows-percent-threshold`](#redundant-rows-percent-threshold-new-in-v757-v854-and-v900), TiKV triggers automatic compaction. + This configuration item takes effect only when [Compaction Filter](/garbage-collection-configuration.md) is enabled. + Default value: `50000` + Minimum value: `0` -### `redundant-rows-percent-threshold` New in v7.5.7 and v9.0.0 +### `redundant-rows-percent-threshold` New in v7.5.7, v8.5.4, and v9.0.0 -+ The percentage of redundant MVCC rows required to trigger TiKV automatic compaction. Redundant rows include RocksDB tombstones, TiKV stale versions, and TiKV deletion tombstones. When the number of redundant MVCC rows reaches [`redundant-rows-threshold`](#redundant-rows-threshold-new-in-v757-and-v900), or when the percentage of these rows reaches `redundant-rows-percent-threshold`, TiKV triggers automatic compaction. ++ The percentage of redundant MVCC rows required to trigger TiKV automatic compaction. Redundant rows include RocksDB tombstones, TiKV stale versions, and TiKV deletion tombstones. When the number of redundant MVCC rows reaches [`redundant-rows-threshold`](#redundant-rows-threshold-new-in-v757-v854-and-v900), or when the percentage of these rows reaches `redundant-rows-percent-threshold`, TiKV triggers automatic compaction. + This configuration item takes effect only when [Compaction Filter](/garbage-collection-configuration.md) is enabled. + Default value: `20` + Minimum value: `0` + Maximum value: `100` -### `bottommost-level-force` New in v7.5.7 and v9.0.0 +### `bottommost-level-force` New in v7.5.7, v8.5.4, and v9.0.0 + Controls whether to force compaction on the bottommost files in RocksDB. + Default value: `true` From ac63a5dad21e05aa862b16eb72f0288704efa1b1 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 31 Dec 2025 11:32:30 +0800 Subject: [PATCH 2/2] Apply suggestions from code review Co-authored-by: Grace Cai --- garbage-collection-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/garbage-collection-configuration.md b/garbage-collection-configuration.md index fe4a6f9e4cf38..81417130d49ee 100644 --- a/garbage-collection-configuration.md +++ b/garbage-collection-configuration.md @@ -117,6 +117,6 @@ show config where type = 'tikv' and name like '%enable-compaction-filter%'; > > - For TiDB versions before v7.1.3, it is recommended to disable Compaction Filter to speed up GC. > - For TiDB versions from v7.1.3 to v7.5.6 and v7.6.0 to v8.5.3, TiDB automatically triggers compaction based on the number of redundant versions in each Region [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and the percentage of redundant versions [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) to improve Compaction Filter GC performance. In this case, adjust these configuration items instead of disabling Compaction Filter. -> - Starting from v7.5.7 and v8.5.4, [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) are deprecated. TiDB now automatically triggers compaction based on [`gc.auto-compaction.redundant-rows-threshold`](/tikv-configuration-file.md#redundant-rows-threshold-new-in-v757-v854-and-v900) and [`gc.auto-compaction.redundant-rows-percent-threshold`](/tikv-configuration-file.md#redundant-rows-percent-threshold-new-in-v757-v854-and-v900). In this case, adjust these configuration items instead of disabling Compaction Filter. +> - Starting from v7.5.7, v8.5.4, and v9.0.0, [`region-compact-min-redundant-rows`](/tikv-configuration-file.md#region-compact-min-redundant-rows-new-in-v710) and [`region-compact-redundant-rows-percent`](/tikv-configuration-file.md#region-compact-redundant-rows-percent-new-in-v710) are deprecated. TiDB now automatically triggers compaction based on [`gc.auto-compaction.redundant-rows-threshold`](/tikv-configuration-file.md#redundant-rows-threshold-new-in-v757-v854-and-v900) and [`gc.auto-compaction.redundant-rows-percent-threshold`](/tikv-configuration-file.md#redundant-rows-percent-threshold-new-in-v757-v854-and-v900). In this case, adjust these configuration items instead of disabling Compaction Filter.