From 3e788a624c7ce61067a0ae4e855890174986e7cc Mon Sep 17 00:00:00 2001 From: zymap Date: Mon, 17 Oct 2022 16:34:41 +0800 Subject: [PATCH] Update website to record the RocksDB configuration changes --- *Motivation* Related PR: #3523 Mailing discussion: https://lists.apache.org/thread/drh4p5prxbcs8gszhxnd1xsv0g48vvbt We introduce the new way to configure the RocksDB but we haven't mentioned it in the website and release-note. This PR add the document for the RocksDB configuration updates. And rename the RocksDB configuration files with the suffix `.default` to avoid loading them by default. *Modification* - Add the configuration changes in the 4.15.0 release notes. - Add how to configure the RocksDB with the new file in the website. - Rename the file with `.default` suffix to avoid loading it by default. --- ...ault_rocksdb.conf => default_rocksdb.conf.default} | 0 ...cksdb.conf => entry_location_rocksdb.conf.default} | 0 ...ksdb.conf => ledger_metadata_rocksdb.conf.default} | 0 site3/website/docs/reference/config.md | 11 +++++++++++ site3/website/src/pages/release-notes.md | 6 ++++++ .../versioned_docs/version-4.15.2/reference/config.md | 6 ++++++ 6 files changed, 23 insertions(+) rename conf/{default_rocksdb.conf => default_rocksdb.conf.default} (100%) rename conf/{entry_location_rocksdb.conf => entry_location_rocksdb.conf.default} (100%) rename conf/{ledger_metadata_rocksdb.conf => ledger_metadata_rocksdb.conf.default} (100%) diff --git a/conf/default_rocksdb.conf b/conf/default_rocksdb.conf.default similarity index 100% rename from conf/default_rocksdb.conf rename to conf/default_rocksdb.conf.default diff --git a/conf/entry_location_rocksdb.conf b/conf/entry_location_rocksdb.conf.default similarity index 100% rename from conf/entry_location_rocksdb.conf rename to conf/entry_location_rocksdb.conf.default diff --git a/conf/ledger_metadata_rocksdb.conf b/conf/ledger_metadata_rocksdb.conf.default similarity index 100% rename from conf/ledger_metadata_rocksdb.conf rename to conf/ledger_metadata_rocksdb.conf.default diff --git a/site3/website/docs/reference/config.md b/site3/website/docs/reference/config.md index 232100a9701..5e528b2fc46 100644 --- a/site3/website/docs/reference/config.md +++ b/site3/website/docs/reference/config.md @@ -226,6 +226,17 @@ The table below lists parameters that you can set to configure bookies. All conf | dbStorage_rocksDB_numFilesInLevel0 | | 10 | | dbStorage_rocksDB_maxSizeInLevel1MB | | 256 | +> Note: +> The RocksDB configuration is deprecated in the bk_server.conf since 4.15.0. It will use independent configuration file. +> There have three configuration file `default_rocksdb.conf`, `entry_location_rocksdb.conf` and `ledger_metadata_rocksdb.conf`. +> `default_rocksdb.conf` is used for default,command until or test case. +> `entry_location_rocksdb.conf` is used for location index, lots of writes and much bigger dataset. +> `ledger_metadata_rocksdb.conf` is used for ledgers db, doesn't need particular configuration. +> +> By default, we are still getting rocksDB properties from the `bk_server.conf`. If you want to use the independent +> configuration file for the rocksDB, you can rename the configuration files without the suffix `.default`, then the rocksDB +> used in the storage will load them. + ## Metadata Service Settings diff --git a/site3/website/src/pages/release-notes.md b/site3/website/src/pages/release-notes.md index 28a383b2b5e..ad90ab26cb6 100644 --- a/site3/website/src/pages/release-notes.md +++ b/site3/website/src/pages/release-notes.md @@ -125,6 +125,12 @@ below. Code that implements the interfaces will need changes. See [PR 2901](https://github.com/apache/bookkeeper/pull/2901) for details. +* RocksDB configuration moves to the independent configuration files. + We used to place the RocksDB configuration properties in the `bk_server.conf`. Now it moved to the independent files + `entry_location_rocksdb.conf` and `ledger_metadata_rocksdb.conf`. + The existing configuration in the `bk_server.conf` for the RocksDB will invalidate. + See [PR 3056](https://github.com/apache/bookkeeper/pull/3056) for details. + ### Highlights #### Configuration diff --git a/site3/website/versioned_docs/version-4.15.2/reference/config.md b/site3/website/versioned_docs/version-4.15.2/reference/config.md index 891b447421e..c49c2cb35b4 100644 --- a/site3/website/versioned_docs/version-4.15.2/reference/config.md +++ b/site3/website/versioned_docs/version-4.15.2/reference/config.md @@ -141,6 +141,12 @@ The table below lists parameters that you can set to configure bookies. All conf | flushInterval | When entryLogPerLedgerEnabled is enabled, checkpoint doesn't happens when a new active entrylog is created / previous one is rolled over. Instead SyncThread checkpoints periodically with 'flushInterval' delay (in milliseconds) in between executions. Checkpoint flushes both ledger entryLogs and ledger index pages to disk. Flushing entrylog and index files will introduce much random disk I/O. If separating journal dir and ledger dirs each on different devices, flushing would not affect performance. But if putting journal dir and ledger dirs on same device, performance degrade significantly on too frequent flushing. You can consider increment flush interval to get better performance, but you need to pay more time on bookie server restart after failure. This config is used only when entryLogPerLedgerEnabled is enabled. | 10000 | | allowStorageExpansion | Allow the expansion of bookie storage capacity. Newly added ledger and index directories must be empty. | false | +> Note: +> The RocksDB configuration is deprecated in the bk_server.conf since 4.15.0. It will use independent configuration file. +> There have three configuration file `default_rocksdb.conf`, `entry_location_rocksdb.conf` and `ledger_metadata_rocksdb.conf`. +> `default_rocksdb.conf` is used for default,command until or test case. +> `entry_location_rocksdb.conf` is used for location index, lots of writes and much bigger dataset. +> `ledger_metadata_rocksdb.conf` is used for ledgers db, doesn't need particular configuration. ## Entry log settings