diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index b95e68d159a83..10b8de7c3ab5d 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -1,6 +1,301 @@ ## Apache Pulsar Release Notes +### 2.10.0 +#### 2022-04-13 + +### Important notice +- Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) +- Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) +- Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) +- Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) +- [PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) +- [PIP 119] Enable consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) +- [PIP 120] Enable client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +- Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) +- [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) +- Fix netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) + +### PIPs +- [PIP 79] Add lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) +- [PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) +- [PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) +- [PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) +- [PIP 104] Add new consumer type: TableView [#12838](https://github.com/apache/pulsar/pull/12838) +- [PIP-105] Support pluggable entry filter in Dispatcher [#12869](https://github.com/apache/pulsar/pull/12869) [#12970](https://github.com/apache/pulsar/pull/12970) [#12979](https://github.com/apache/pulsar/pull/12979) +- [PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility [#12536](https://github.com/apache/pulsar/pull/12536) +- [PIP 107] Introduce chunk message ID [#12403](https://github.com/apache/pulsar/pull/12403) +- [PIP 110] Support Topic metadata - PART-1 create topic with properties [#12818](https://github.com/apache/pulsar/pull/12818) +- [PIP 121] Pulsar cluster level auto failover on client side [#13316](https://github.com/apache/pulsar/pull/13316) +- [PIP-124] Create init subscription before sending message to DLQ [#13355](https://github.com/apache/pulsar/pull/13355) +- [PIP-130] Apply redelivery backoff policy for ack timeout [#13707](https://github.com/apache/pulsar/pull/13707) +- [PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set [#13599](https://github.com/apache/pulsar/pull/13599) +- [PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) + +### Broker +- [PIP 45] Pluggable metadata interface + - Add BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) + - Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) + - Convert BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) + - Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) + - Remove old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) + - Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) + - Remove ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) + - Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) + - Allow configuring metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) + - Allow configuring metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) + - Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) + - Allow configuring configurationMetadataStore when initializing cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) + - Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) + - Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) + - Deprecate zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) + - Fix metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) +- Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) +- Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) +- Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) +- Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) +- Support splitting the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) +- Support creating a subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) +- Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) +- Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) +- Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) +- Add uniform load shedder strategy to distribute traffic uniformly across brokers [#12902](https://github.com/apache/pulsar/pull/12902) +- Provide option to split bundle based on load [#12378](https://github.com/apache/pulsar/pull/12378) +- Use AuthorizationService#grantPermissionAsync to grant topic permission [#12515](https://github.com/apache/pulsar/pull/12515) +- Only store authentication data after authentication is complete [#12077](https://github.com/apache/pulsar/pull/12077) +- Allow to have different instances LocalMemoryMetadataStore that share the same state [#12390](https://github.com/apache/pulsar/pull/12390) +- Allow `GetTopicsOfNamespace` op with `consume` permission [#12600](https://github.com/apache/pulsar/pull/12600) +- Add local filesystem backend for package manager [#12708](https://github.com/apache/pulsar/pull/12708) +- Add stop replicator producer logic when start replicator cluster failed [#12724](https://github.com/apache/pulsar/pull/12724) +- Apply PolicyHierarchyValue to inactiveTopicPolicies [#12687](https://github.com/apache/pulsar/pull/12687) +- Fix can not get leader broker in follower brokers [#11353](https://github.com/apache/pulsar/pull/11353) +- Fix broker gc log options [#11285](https://github.com/apache/pulsar/pull/11285) +- Fix NPE of ZkBookieRackAffinityMapping [#11947](https://github.com/apache/pulsar/pull/11947) +- Fix prefix setting in JWT authn and avoid multi calls for the getProperty [#12132](https://github.com/apache/pulsar/pull/12132) +- Fix missed check exit code of stop before calling start [#12368](https://github.com/apache/pulsar/pull/12368) +- Fix getting children of parent nodes in LocalMemoryMetadataStore [#12491](https://github.com/apache/pulsar/pull/12491) +- Fix collection get bug in ResourceGroupService [#12499](https://github.com/apache/pulsar/pull/12499) +- Fix deadlock in metadata-store callback thread [#12753](https://github.com/apache/pulsar/pull/12753) +- Improve exceptions thrown when handling the schema resource [#12155](https://github.com/apache/pulsar/pull/12155) +- Trim topic name [#12453](https://github.com/apache/pulsar/pull/12453) +- Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic [#12658](https://github.com/apache/pulsar/pull/12658) +- Optimize isValidPath check in MetadataStore [#12663](https://github.com/apache/pulsar/pull/12663) +- Close connection after receiving unexpected SendCommand [#12780](https://github.com/apache/pulsar/pull/12780) +- Ensure cache is refreshed (and not just invalidated) after a store write [#12788](https://github.com/apache/pulsar/pull/12788) +- Optimize topicMaxMessageSize with topic local cache. [#12830](https://github.com/apache/pulsar/pull/12830) +- Optimize blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer [#12874](https://github.com/apache/pulsar/pull/12874) +- Only refresh metadata if path is already in cache after write. [#12896](https://github.com/apache/pulsar/pull/12896) +- Optimize put and get methods in AbstractMetadataStore [#12916](https://github.com/apache/pulsar/pull/12916) +- Fix zk-node leak of admin path [#12972](https://github.com/apache/pulsar/pull/12972) +- Optimize TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies [#13000](https://github.com/apache/pulsar/pull/13000) +- Fix race condition in FaultInjectionMetadataStore#programmedFailure [#13007](https://github.com/apache/pulsar/pull/13007) +- Fix usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts [#13025](https://github.com/apache/pulsar/pull/13025) +- Consider topics in pulsar/system namespace as system topics [#13050](https://github.com/apache/pulsar/pull/13050) +- Fix wrong result for looking up a non-exist topic by rest api [#13055](https://github.com/apache/pulsar/pull/13055) +- Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes [#13063](https://github.com/apache/pulsar/pull/13063) +- Make load-balancer config dynamic for the runtime tuning [#13074](https://github.com/apache/pulsar/pull/13074) +- Optimize TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https://github.com/apache/pulsar/pull/13082) +- Load balancer supports disabling max-session for bundle split [#13108](https://github.com/apache/pulsar/pull/13108) +- Optimize TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies [#13121](https://github.com/apache/pulsar/pull/13121) +- Log thread dump when Zookeeper session expires to help detect possible deadlocks [#13124](https://github.com/apache/pulsar/pull/13124) +- Add removeMaxConsumersPerSubscription method for v1 namespace [#13192](https://github.com/apache/pulsar/pull/13192) +- Fixe error value for 1MB in metrics [#13213](https://github.com/apache/pulsar/pull/13213) +- Optimize TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies [#13241](https://github.com/apache/pulsar/pull/13241) +- Add schema compatibility strategy on topic level [#13297](https://github.com/apache/pulsar/pull/13297) +- Optimize TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies [#13361](https://github.com/apache/pulsar/pull/13361) +- Optimize TopicPolicies#replicationClusters with HierarchyTopicPolicies [#13495](https://github.com/apache/pulsar/pull/13495) +- Fix compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) +- Fix race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) +- Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) +- Return null instead of RestException when getting bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) +- Fix failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) +- Optimize TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) +- Fix raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) +- Optimize TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies [#13618](https://github.com/apache/pulsar/pull/13618) +- Optimize TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies [#13649](https://github.com/apache/pulsar/pull/13649) +- Optimize TopicPolicies#compactionThreshold with HierarchyTopicPolicies [#13710](https://github.com/apache/pulsar/pull/13710) +- Add deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) +- Fix NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) +- Optimize deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) +- Use shared executors for broker and geo-replication clients [#13839](https://github.com/apache/pulsar/pull/13839) +- Fix call sync method in async rest api for + - for internalDeletePartitionedTopic [#13805](https://github.com/apache/pulsar/pull/13805) + - for internalCreateSubscription. [#13873](https://github.com/apache/pulsar/pull/13873) + - for internalGetBacklogSizeByMessageId [#13871](https://github.com/apache/pulsar/pull/13871) + - for internalTriggerCompaction [#13853](https://github.com/apache/pulsar/pull/13853) + - for internalGetManagedLedgerInfo [#13847](https://github.com/apache/pulsar/pull/13847) + - for internalGetSubscriptions [#13846](https://github.com/apache/pulsar/pull/13846) + - for internalUnloadTopic [#13845](https://github.com/apache/pulsar/pull/13845) + - for internalGetLastMessageId [#13882](https://github.com/apache/pulsar/pull/13882) + - for internalExpireMessagesByTimestamp [#13880](https://github.com/apache/pulsar/pull/13880) + - for internalResetCursorOnPosition [#13879](https://github.com/apache/pulsar/pull/13879) + - for internalExpireMessagesByPosition [#13878](https://github.com/apache/pulsar/pull/13878) + - for internalGetMessageById. [#13876](https://github.com/apache/pulsar/pull/13876) + - for internalSetReplicatedSubscriptionStatus [#13887](https://github.com/apache/pulsar/pull/13887) + - for internalGetPartitionedStats [#13886](https://github.com/apache/pulsar/pull/13886) + - for onPoliciesUpdate method [#13885](https://github.com/apache/pulsar/pull/13885) + - for internalGetPartitionedStatsInternal [#13884](https://github.com/apache/pulsar/pull/13884) + - for internalTerminatePartitionedTopic [#13890](https://github.com/apache/pulsar/pull/13890) + - for internalSkipAllMessages [#13901](https://github.com/apache/pulsar/pull/13901) + - for preValidation [#13962](https://github.com/apache/pulsar/pull/13962) + - for internalSetReplicationClusters and internalRemoveReplicationClusters [#13961](https://github.com/apache/pulsar/pull/13961) +- Make PulsarAuthorizationProvider#grantPermissionAsync actually async [#13897](https://github.com/apache/pulsar/pull/13897) +- Fix no value present when updating dispatch rate [#13900](https://github.com/apache/pulsar/pull/13900) +- Optimize retentionPolicies with HierarchyTopicPolicies [#13909](https://github.com/apache/pulsar/pull/13909) +- Improve 403 response message wih consume permission on namespace [#13912](https://github.com/apache/pulsar/pull/13912) +- Fix arithmetic exception for uniform load shedder [#13914](https://github.com/apache/pulsar/pull/13914) +- Add threshold for each unload round for uniform load shedder [#13915](https://github.com/apache/pulsar/pull/13915) +- Fix call sync method in async rest api for internalDeleteSubscription #13884 [#13917](https://github.com/apache/pulsar/pull/13917) +- Change ``BrokersBase`` api ``getActiveBrokers`` and ``getLeaderBroker`` to pure async. [#13935](https://github.com/apache/pulsar/pull/13935) +- Do not create missing topic when loading namespace [#13948](https://github.com/apache/pulsar/pull/13948) +- Only reply to client when code completes producerFuture [#13949](https://github.com/apache/pulsar/pull/13949) +- Improve error logging for topic not found [#13950](https://github.com/apache/pulsar/pull/13950) +- Make validateTenantOperation method async in PulsarWebResource [#14008](https://github.com/apache/pulsar/pull/14008) +- Optimize maxUnackedMessagesOnSubscription with HierarchyTopicPolicies [#14011](https://github.com/apache/pulsar/pull/14011) +- Set default value of applied to false in getSchemaCompatibilityStrategy [#14012](https://github.com/apache/pulsar/pull/14012) +- Make validateTopicPolicyOperation method async in PulsarWebResource [#14024](https://github.com/apache/pulsar/pull/14024) +- Fix producerFuture.completeExceptionally not called before sendErrorResponse [#14025](https://github.com/apache/pulsar/pull/14025) +- Make triggerOffload method async [#14027](https://github.com/apache/pulsar/pull/14027) +- Make offloadStatus method async [#14029](https://github.com/apache/pulsar/pull/14029) +- Make validateNamespacePolicyOperation method async in PulsarWebResource [#14033](https://github.com/apache/pulsar/pull/14033) +- Make internalSkipMessages method async [#14045](https://github.com/apache/pulsar/pull/14045) +- Make PersistentTopicsBase#internalSetBacklogQuota async [#14051](https://github.com/apache/pulsar/pull/14051) +- Adjust the validation for policy schemaCompatibilityStrategy [#14061](https://github.com/apache/pulsar/pull/14061) +- Remove Persistent Topics v3 API - use custom media type instead [#14117](https://github.com/apache/pulsar/pull/14117) +- Fix race condition in PulsarLedgerIdGenerator#generateShortLedgerId [#14118](https://github.com/apache/pulsar/pull/14118) +- Optimize topic policy with HierarchyTopicPolicies about subscriptionDispatchRate [#14151](https://github.com/apache/pulsar/pull/14151) +- Make `BrokerBase#deleteDynamicConfiguration` to pure async method [#14163](https://github.com/apache/pulsar/pull/14163) +- Set default value of applied to false on topic policy [#14181](https://github.com/apache/pulsar/pull/14181) +- Change broker producer fence log level [#14196](https://github.com/apache/pulsar/pull/14196) +- Remove duplicated filter for UniformLoadShedder#findBundlesForUnloading [#14198](https://github.com/apache/pulsar/pull/14198) +- Adjust topic exists check logic in http lookup process [#14199](https://github.com/apache/pulsar/pull/14199) +- Fix NPE of internalExpireMessagesByTimestamp [#14243](https://github.com/apache/pulsar/pull/14243) +- Fix rackaware placement policy does not take effect after delete rack configuration [#14248](https://github.com/apache/pulsar/pull/14248) +- Fix print error log when server return redirect (http code 307) [#14259](https://github.com/apache/pulsar/pull/14259) +- Optimize topic policy with HierarchyTopicPolicies about publishRate [#14267](https://github.com/apache/pulsar/pull/14267) +- Fix ack-hole and backlog for persistent-replicator [#14282](https://github.com/apache/pulsar/pull/14282) +- Fix NPE in internalSkipMessages [#14297](https://github.com/apache/pulsar/pull/14297) +- Validate blank advertised listener name [#14306](https://github.com/apache/pulsar/pull/14306) +- Let entries expire in the metadata caches [#14154](https://github.com/apache/pulsar/pull/14154) +- Fix transaction system topic loop creation [#12749](https://github.com/apache/pulsar/pull/12749) +- Fix topic transaction buffer handle null snapshot [#12758](https://github.com/apache/pulsar/pull/12758) +- Optimize changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) + +### Clients +- [Java] Support creating a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) +- [Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) +- [Java] Fix the producer OOM if got an exception while adding messages to batch container [#12170](https://github.com/apache/pulsar/pull/12170) +- [Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) +- [Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) +- [Java] Fix confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) +- [Java] Fix parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) +- [Java] Add getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) +- [Java] Allow config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) +- [Java] Support adding custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) +- [Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) +- [Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) +- [Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) +- [Java] Log producer batchSize and msgSize percentiles [#14229](https://github.com/apache/pulsar/pull/14229) +- [C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) +- [C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) +- [C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) +- [C++] Support more pulsar command names and versions in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) +- [C++] Optimize MessageBuilder and SharedBuffer to avoid unnecessary memory copy [#13293](https://github.com/apache/pulsar/pull/13293) +- [C++] Fix in Apple Silicon macOS the clang-format cannot find [#13333](https://github.com/apache/pulsar/pull/13333) +- [C++] Add clang-format check for Pulsar Wireshark dissector [#13349](https://github.com/apache/pulsar/pull/13349) +- [C++] Adjust clang-format search names [#13369](https://github.com/apache/pulsar/pull/13369) +- [C++] Fix Wireshark dissector decode send command metadata behavior [#13471](https://github.com/apache/pulsar/pull/13471) +- [C++] PIP 37: Support large message size [#13627](https://github.com/apache/pulsar/pull/13627) +- [C++] Fix the consumer configuration inconsistency with Java client [#14070](https://github.com/apache/pulsar/pull/14070) +- [Python] Provide __str__ operator for BytesSchema [#12593](https://github.com/apache/pulsar/pull/12593) +- [Python] Support is_connected in Python Client [#13662](https://github.com/apache/pulsar/pull/13662) +- [C] Add missing includes in reader_configuration.h [#12966](https://github.com/apache/pulsar/pull/12966) +- [C] Add pulsar_client_subscribe_multi_topics and pulsar_client_subscribe_pattern [#12965](https://github.com/apache/pulsar/pull/12965) + +### Pulsar IO and Pulsar Functions + +- [Functions] Prevent NPE while stopping a non started Pulsar LogAppender [#12643](https://github.com/apache/pulsar/pull/12643) +- [Functions] Allow configuring different implementations for Pulsar functions state store [#12646](https://github.com/apache/pulsar/pull/12646) +- [Functions] Clean os even when statusFuture complete exceptionally [#12767](https://github.com/apache/pulsar/pull/12767) +- [Functions] Override inactive_topic_policies in Pulsar Functions namespace creation [#13048](https://github.com/apache/pulsar/pull/13048) +- [Functions] Add possibility to pass additional JVM arguments to the function JVM (additionalJavaRuntimeArguments) [#13282](https://github.com/apache/pulsar/pull/13282) +- [Functions] Fix `getTlsTrustChainBytes` not work when functions worker not run with broker [#13875](https://github.com/apache/pulsar/pull/13875) +- [Functions] Fix distributed log metadata not correctly initialized [#13891](https://github.com/apache/pulsar/pull/13891) +- [Functions] Remove sensitive information from log [#14159](https://github.com/apache/pulsar/pull/14159) +- [IO Connector] Pass client builder if no service url provided to debezium connector [#12145](https://github.com/apache/pulsar/pull/12145) +- [IO Connector] Fix: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test [12374](https://github.com/apache/pulsar/pull/12374) +- [IO Connector] Fix the marshal and unmarshal the sink config [#12625](https://github.com/apache/pulsar/pull/12625) +- [IO Connector] Support renaming the file that has been processed [#13373](https://github.com/apache/pulsar/pull/13373) +- [IO Connector] Support Fixed and ENUM datatypes for ElasticSearch Sink [#13800](https://github.com/apache/pulsar/pull/13800) +- [IO Connector] Implement --retain-key-ordering (KEY_SHARED subscription) for Sinks [#14083](https://github.com/apache/pulsar/pull/14083) + +### Observability + +- [Broker] Expose broker bundles metrics to prometheus [#12366](https://github.com/apache/pulsar/pull/12366) +- [Broker] Add publishRateLimitedTimes to topic metrics [#13538](https://github.com/apache/pulsar/pull/13538) +- [Broker] Fix bundle metrics would overwrite loadbalance metrics [#13641](https://github.com/apache/pulsar/pull/13641) +- [Broker] Fix managed cursor acknowledgment state metric names [#13844](https://github.com/apache/pulsar/pull/13844) +- [Java Client] Add pending-queue size metrics to producer stats [#12674](https://github.com/apache/pulsar/pull/12674) + +### CLI +- [Pulsar Admin] Support a mapping from brokers to partitioned topics that each broker serves [#11763](https://github.com/apache/pulsar/pull/11763) +- [Pulsar Admin] Support the termination of a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) +- [Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) +- [Pulsar Admin] Add remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) +- [Pulsar Admin] Fix output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) +- [Pulsar Admin] Add a metric to get the earliest time in the backlog [#12523](https://github.com/apache/pulsar/pull/12523) +- [Pulsar Admin] Reduce severity of log "refreshing key manager" in KeyManagerProxy [#12594](https://github.com/apache/pulsar/pull/12594) +- [Pulsar Admin] Support getting a list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) +- [Pulsar Admin] Add get-replicated-subscription-status command for topic [#12891](https://github.com/apache/pulsar/pull/12891) +- [Pulsar Admin] Add remove-subscription-types-enabled command for topic [#12983](https://github.com/apache/pulsar/pull/12983) +- [Pulsar Admin] Validate the size options in cmd for topic and namespace [#13002](https://github.com/apache/pulsar/pull/13002) +- [Pulsar Admin] Support subscription across multiple clusters [#13482](https://github.com/apache/pulsar/pull/13482) +- [Pulsar Admin] Support deduplication across multiple clusters [#13487](https://github.com/apache/pulsar/pull/13487) +- [Pulsar Admin] Support message TTL across multiple clusters [#13484](https://github.com/apache/pulsar/pull/13484) +- [Pulsar Admin] Support persistence policies across multiple clusters [#13483](https://github.com/apache/pulsar/pull/13483) +- [Pulsar Admin] Support publish rate across multiple clusters [#13496](https://github.com/apache/pulsar/pull/13496) +- [Pulsar Admin] Support max consumers across multiple clusters [#13521](https://github.com/apache/pulsar/pull/13521) +- [Pulsar Admin] Support max producer across multiple clusters [#13519](https://github.com/apache/pulsar/pull/13519) +- [Pulsar Admin] Support compaction threshold across multiple clusters [#13513](https://github.com/apache/pulsar/pull/13513) +- [Pulsar Admin] Support offload policies across multiple clusters [#13534](https://github.com/apache/pulsar/pull/13534) +- [Pulsar Admin] Support max unacked messages per consumer across multiple clusters [#13547](https://github.com/apache/pulsar/pull/13547) +- [Pulsar Admin] Support max message size support across multiple clusters [#13579](https://github.com/apache/pulsar/pull/13579) +- [Pulsar Admin] Support deduplication snapshot interval across multiple clusters [#13578](https://github.com/apache/pulsar/pull/13578) +- [Pulsar Admin] Support delayed delivery policy across multiple clusters [#13550](https://github.com/apache/pulsar/pull/13550) +- [Pulsar Admin] Support max unacked messages on subscription across multiple clusters [#13549](https://github.com/apache/pulsar/pull/13549) +- [Pulsar Admin] Support replicator dispatch rate across multiple clusters [#13624](https://github.com/apache/pulsar/pull/13624) +- [Pulsar Admin] Support max subscriptions per topic across multiple clusters [#13623](https://github.com/apache/pulsar/pull/13623) +- [Pulsar Admin] Move schema compatibility strategy cmd from topics to topicPolicies [#14225](https://github.com/apache/pulsar/pull/14225) +- [Pulsar CLI] Add restart command to pulsar-daemon [#12279](https://github.com/apache/pulsar/pull/12279) +- [Pulsar CLI] Print log for standalone when configuration is failed to load [#12280](https://github.com/apache/pulsar/pull/12280) +- [Client Tool] Support Disabling Replication [#13659](https://github.com/apache/pulsar/pull/13659) +- [Perf Tool] Add auth for transaction perf [#14271](https://github.com/apache/pulsar/pull/14271) + +### Others +- [Proxy] Fix Pulsar Proxy to re-use authentication instance [#12245](https://github.com/apache/pulsar/pull/12245) +- [Proxy] Fix auto-cert refresh when proxy connects to broker [#14130](https://github.com/apache/pulsar/pull/14130) +- [WebSocket] Fix the batch message ack [#12530](https://github.com/apache/pulsar/pull/12530) +- [Package Management] check service status before run commands [#12847](https://github.com/apache/pulsar/pull/12847) +- [Package Management] Add package management filesystem storage into the distribution lib [#13202](https://github.com/apache/pulsar/pull/13202) +- [Package Management] Fix the filesystem storage failure [#13218](https://github.com/apache/pulsar/pull/13218) +- [Schema] Fix pulsar use json or avro primitive schema [#12886](https://github.com/apache/pulsar/pull/12886) +- [Tiered Storage] Fix FileSystemManagedLedgerOffloader can not cleanup outdated ledger [#12309](https://github.com/apache/pulsar/pull/12309) +- [Tiered Storage] Fix NoClassDefFoundError: com/google/inject/AbstractModule in pulsar-io/batch-data-generator and Jcloud offloader [#14150](https://github.com/apache/pulsar/pull/14150) +- Add lua wireshark for pulsar [#13564](https://github.com/apache/pulsar/pull/13564) +- Fix missing new line at the end of proxy.conf [#14359](https://github.com/apache/pulsar/pull/14359) + +### Library updates + +- Upgrade function's Go client version to v0.7.0 [#12839](https://github.com/apache/pulsar/pull/12839) +- Upgrade dependencies (guava and what brought in older guava) to get rid of the guava-related CVE-2018-10237 and CVE-2020-8908 [#13716](https://github.com/apache/pulsar/pull/13716) +- Use dependencyManagement to enforce snakeyaml version to 1.30 [#13722](https://github.com/apache/pulsar/pull/13722) +- Upgrade dependencies to get rid of pulsar-io/jdbc related CVE-2020-13692 [#13753](https://github.com/apache/pulsar/pull/13753) +- Remove --illegal-access errors resulting from Google Guice (upgrade to 5.0.1 and JClouds to 2.4.0) [#13810](https://github.com/apache/pulsar/pull/13810) +- Upgrade jakarta.el to 3.0.4 to get rid of CVE-2021-28170 [#13943](https://github.com/apache/pulsar/pull/13943) +- Upgrade Netty to 4.1.73.Final [#13981](https://github.com/apache/pulsar/pull/13981) +- Remove net.jodah.failsafe dependency (fix JDK17 build) [#14124](https://github.com/apache/pulsar/pull/14124) +- Upgrade netty version to 4.1.74.Final [#14257](https://github.com/apache/pulsar/pull/14257) + ### 2.8.3 #### 2022-03-21