Skip to content

[docs] Update configuration information for 2.4.x releases #6047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sijie merged 2 commits into from
Jan 18, 2020
Merged

[docs] Update configuration information for 2.4.x releases #6047

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1202198
update config parameters
Jennifer88huang-zz Jan 14, 2020
4283f6e
update the information in 2.4.x releases
Jennifer88huang-zz Jan 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion site2/docs/reference-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,13 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a deduplication informational snapshot is taken. A larger interval will lead to fewer snapshots being taken, though this would also lengthen the topic recovery time (the time required for entries published after the snapshot to be replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity (in minutes) after which the broker will discard deduplication information related to a disconnected producer. |360|
|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second dispatch throttling-limit for every replicator in replication. The value of `0` means disabling replication message dispatch-throttling| 0 |
|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second dispatch throttling-limit for every replicator in replication. The value of `0` means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds |30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have reached the quota |60|
|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
|allowAutoTopicCreation| Enable topic auto creation if a new producer or consumer connected |true|
|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned) that is allowed to be automatically created. |Partitioned|
|defaultNumPartitions| The number of partitioned topics that is allowed to be automatically created if `allowAutoTopicCreationType` is partitioned |1|
Expand Down Expand Up @@ -224,6 +226,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|loadManagerClassName| Name of load manager to use |org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for ledger offloading |2|
|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache unacknowledged messages |true|
|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully offloaded to long term storage and the ledger being deleted from bookkeeper | 14400000|
|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
Expand Down Expand Up @@ -360,11 +363,18 @@ The [`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation group if bookkeeperClientIsolationGroups doesn't have enough bookie available. ||
|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that should be available as part of bookkeeperClientIsolationGroups else broker will include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency for the managed ledger cache (evictions/sec) | 100.0 |
|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed in cache for more than the configured time, will be evicted | 1000 |
|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged' and thus should be set as inactive. | 1000|
|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
Expand Down
1 change: 1 addition & 0 deletions site2/docs/reference-metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ All the topic metrics are labelled with the following labels:
| pulsar_storage_size | Gauge | The total storage size of the topics in this topic owned by this broker (bytes). |
| pulsar_storage_backlog_size | Gauge | The total backlog size of the topics of this topic owned by this broker (messages). |
| pulsar_storage_offloaded_size | Gauge | The total amount of the data in this topic offloaded to the tiered storage (bytes). |
| pulsar_storage_backlog_quota_limit | Gauge | The total amount of the data in this topic that limit the backlog quota (bytes). |
| pulsar_storage_write_rate | Gauge | The total message batches (entries) written to the storage for this topic (message batches / second). |
| pulsar_storage_read_rate | Gauge | The total message batches (entries) read from the storage for this topic (message batches / second). |
| pulsar_subscription_delayed | Gauge | The total message batches (entries) are delayed for dispatching. |
Expand Down
38 changes: 28 additions & 10 deletions site2/website/versioned_docs/version-2.4.0/reference-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ sidebar_label: Pulsar configuration
original_id: reference-configuration
---


<style type="text/css">
table{
font-size: 80%;
Expand All @@ -14,15 +15,16 @@ original_id: reference-configuration

Pulsar configuration can be managed either via a series of configuration files contained in the [`conf`](https://github.com/apache/pulsar/tree/master/conf) directory of a Pulsar [installation](getting-started-standalone.md)

* [BookKeeper](#bookkeeper)
* [Broker](#broker)
* [Client](#client)
* [Service discovery](#service-discovery)
* [Log4j](#log4j)
* [Log4j shell](#log4j-shell)
* [Standalone](#standalone)
* [WebSocket](#websocket)
* [ZooKeeper](#zookeeper)
- [BookKeeper](#bookkeeper)
- [Broker](#broker)
- [Client](#client)
- [Service discovery](#service-discovery)
- [Log4j](#log4j)
- [Log4j shell](#log4j-shell)
- [Standalone](#standalone)
- [WebSocket](#websocket)
- [Pulsar proxy](#pulsar-proxy)
- [ZooKeeper](#zookeeper)

## BookKeeper

Expand Down Expand Up @@ -122,11 +124,16 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a deduplication informational snapshot is taken. A larger interval will lead to fewer snapshots being taken, though this would also lengthen the topic recovery time (the time required for entries published after the snapshot to be replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity (in minutes) after which the broker will discard deduplication information related to a disconnected producer. |360|
|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second dispatch throttling-limit for every replicator in replication. The value of `0` means disabling replication message dispatch-throttling| 0 |
|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second dispatch throttling-limit for every replicator in replication. The value of `0` means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds |30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have reached the quota |60|
|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
|allowAutoTopicCreation| Enable topic auto creation if a new producer or consumer connected |true|
|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned) that is allowed to be automatically created. |Partitioned|
|defaultNumPartitions| The number of partitioned topics that is allowed to be automatically created if `allowAutoTopicCreationType` is partitioned |1|
|brokerDeleteInactiveTopicsEnabled| Enable the deletion of inactive topics |true|
|brokerDeleteInactiveTopicsFrequencySeconds| How often to check for inactive topics |60|
|messageExpiryCheckIntervalInMinutes| How frequently to proactively check and purge expired messages |5|
Expand Down Expand Up @@ -221,6 +228,9 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|loadManagerClassName| Name of load manager to use |org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for ledger offloading |2|
|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache unacknowledged messages |true|
|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully offloaded to long term storage and the ledger being deleted from bookkeeper | 14400000|
|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
|s3ManagedLedgerOffloadBucket| For Amazon S3 ledger offload, Bucket to place offloaded ledger into ||
|s3ManagedLedgerOffloadServiceEndpoint| For Amazon S3 ledger offload, Alternative endpoint to connect to (useful for testing) ||
Expand Down Expand Up @@ -355,11 +365,18 @@ The [`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation group if bookkeeperClientIsolationGroups doesn't have enough bookie available. ||
|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that should be available as part of bookkeeperClientIsolationGroups else broker will include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency for the managed ledger cache (evictions/sec) | 100.0 |
|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed in cache for more than the configured time, will be evicted | 1000 |
|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged' and thus should be set as inactive. | 1000|
|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
Expand Down Expand Up @@ -437,6 +454,7 @@ The [Pulsar proxy](concepts-architecture-overview.md#pulsar-proxy) can be config
|servicePortTls| The port to use to server binary Protobuf TLS requests |6651|
|statusFilePath| Path for the file used to determine the rotation status for the proxy instance when responding to service discovery health checks ||
|authenticationEnabled| Whether authentication is enabled for the Pulsar proxy |false|
|authenticateMetricsEndpoint| Whether the '/metrics' endpoint requires authentication. Defaults to true. 'authenticationEnabled' must also be set for this to take effect. |true|
|authenticationProviders| Authentication provider name list (a comma-separated list of class names) ||
|authorizationEnabled| Whether authorization is enforced by the Pulsar proxy |false|
|authorizationProvider| Authorization provider as a fully qualified class name |org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider|
Expand Down
Loading