From 1b5e1b72d6833a728898ca7b482221cda15e6b3d Mon Sep 17 00:00:00 2001 From: Michael Marshall Date: Thu, 17 Feb 2022 06:51:39 -0600 Subject: [PATCH 01/20] Make Docker images non-root, by default, and OpenShift compliant (#13376) Master Issue: https://github.com/apache/pulsar/issues/11269 ### Motivation In order to increase the overall security of our Pulsar docker images, they should default to run as the non-root user. While updating these permissions, I make sure to comply with the OpenShift spec so the docker image can run on that platform out of the box. Once we finalize these changes, we will need to update the Apache Pulsar Helm chart to make sure that deployments take advantage of this feature. We'll use the `fsGroup` to make sure that k8s sets the appropriate file system permissions for the zookeeper, bookkeeper, and function pods. ### Modifications * Default to run as UID 10000. As noted in the `Dockerfile`, this UID is arbitrary. No logic should rely on this id. * Update filesystem permissions so that the group user has sufficient write permission. The group user is 0 (root). * Remove unnecessary write access. * The `/pulsar/{conf,data,logs}` directories and their members must be writable by the root group. I don't know of any other directories that need to be written to. Note that the `bin/pulsar-admin` too creates a log file in the `/pulsar/logs` directory. Please let me know if there are any additional * Note also that the executable file permissions are already set in our git repo. Those permissions are inherited by the docker image when we run the `COPY` directive in the `Dockerfile`. * There are no changes to the function worker in the k8s runtime. We do not need them because we already merged https://github.com/apache/pulsar/commit/04b5da0f95794259694cc781e8960b7e52fac06b. * Add note to `conf/bkenv.sh`, as it is a `.sh` script that is not executable (and doesn't need to be). * Update test docker image and `supervisord` configuration. Note: it's unclear to me how the OpenShift spec handles restarts. I know that the UID is arbitrary. It's possible that the umask needs to be switched from `022` to `002`. Setting the umask in the docker image does not persist for consumers of the image, so this would need to be set in a helm chart. ### Verifying this change You can access a test image built with these changes here: `michaelmarshall/pulsar:2.10.0-SNAPSHOT`. I have already run some manual tests like `bin/pulsar standalone` in the container. I still need to deploy an actual cluster to verify that all of the unique components work correctly. Because we already merged https://github.com/apache/pulsar/commit/04b5da0f95794259694cc781e8960b7e52fac06b, the upgrade scenarios are already simplified. If this change is in 2.10.0, that means 2.8 and 2.9 will be compatible for certain function worker upgrade scenarios. I wrote test criteria in https://github.com/apache/pulsar/issues/11269. I'll need to follow up on that criteria using my newly build image. I should be able to look closer at this tomorrow. We'll also need tests to pass, as I modified some tests with this PR. ### References The following links were useful in understanding how to make these changes: * https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html * https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids ### Does this pull request potentially affect one of the following parts: This PR updates our Docker images in a breaking way. It could result in bookkeepers, zookeepers, or functions with insufficient permissions. We will mitigate these permissions by updating the helm chart. These changes are easily overridden by extending the docker image. In k8s, you can use the pod's `securityContext` to override the user or group. (cherry picked from commit f7f861988780578e2ba102ce4e22fa8841c13e3b) --- conf/bkenv.sh | 2 + docker/README.md | 109 ++++++++++++++++++ docker/pulsar/Dockerfile | 20 +++- site2/docs/getting-started-docker.md | 1 + .../latest-version-image/Dockerfile | 11 ++ .../latest-version-image/conf/bookie.conf | 1 + .../latest-version-image/conf/broker.conf | 2 +- .../conf/functions_worker.conf | 2 +- .../latest-version-image/conf/global-zk.conf | 2 +- .../latest-version-image/conf/local-zk.conf | 2 +- .../conf/presto_worker.conf | 3 +- .../latest-version-image/conf/proxy.conf | 2 +- 12 files changed, 150 insertions(+), 7 deletions(-) create mode 100644 docker/README.md diff --git a/conf/bkenv.sh b/conf/bkenv.sh index d3ef33e07af4a..e6b7af687fdf2 100644 --- a/conf/bkenv.sh +++ b/conf/bkenv.sh @@ -18,6 +18,8 @@ # under the License. # +# NOTE: this script is intentionally not executable. It is only meant to be sourced for environment variables. + # Set JAVA_HOME here to override the environment setting # JAVA_HOME= diff --git a/docker/README.md b/docker/README.md new file mode 100644 index 0000000000000..5318d331ce9f4 --- /dev/null +++ b/docker/README.md @@ -0,0 +1,109 @@ + + +# Apache Pulsar Docker Images + +The Apache Pulsar community produces 2 docker images with each official release. + +* `apachepulsar/pulsar` - contains the necessary components for a working Pulsar cluster +* `apachepulsar/pulsar-all` - extends the `apachepulsar/pulsar` image by adding many Pulsar connectors and offloaders + +Since the 2.10.0 release, these docker images run as an unnamed, non-root user that is also part of the root group, by +default. This was done to increase container security. The user is part of the root group to ensure that the container +image can easily run on OpenShift and to ensure that the Pulsar process can write to configuration files. + +## Development + +You can build and test these docker images on your own machine by running the `./build.sh` script in this directory. +Note that you first must build the project in order to have the right dependencies in your local environment. + +## Building Derivative Custom Images + +If you find the `apachepulsar/pulsar-all` docker image too large, but you want to use a connector or an offloader, +you can easily build an image with a curated list of connectors or offloaders based on the official Apache Pulsar +images. You can use the following sample docker image as a guide: + +```Dockerfile +ARG VERSION + +# Load the pulsar-all image as a builder image +FROM apachepulsar/pulsar-all:${VERSION} as pulsar-all + +FROM apachepulsar/pulsar:${VERSION} + +# Add the cassandra connector +COPY --from=pulsar-all /pulsar/connectors/pulsar-io-cassandra-*.nar /pulsar/connectors + +# Add the jcloud offloader +COPY --from=pulsar-all /pulsar/connectors/tiered-storage-jcloud-*.nar /pulsar/offloaders +``` + +NOTE: the above example uses a wildcard in the `COPY` commands because argument expansion does not work for `COPY`. + +Assuming that you have the above `Dockerfile` in your local directory and are running docker on your local host, you can +run the following command to build a custom image with the cassandra connector and the jcloud offloader. + +```shell +docker build --build-arg VERSION=2.9.1 -t pulsar-custom:2.9.1 . +``` + +For reference, here are the sizes of the official 2.9.1 docker images and the custom image built from the above +`Dockerfile`: + +| REPOSITORY | TAG | SIZE | +| :---------------------- | :---- | :----- | +| apachepulsar/pulsar | 2.9.1 | 1.59GB | +| apachepulsar/pulsar-all | 2.9.1 | 3.44GB | +| pulsar-custom | 2.9.1 | 1.6GB | + + +## Troubleshooting non-root containers + +Troubleshooting is harder because the docker image runs as a non-root user. For example, a non-root user won't be able +to download arbitrary utilities. There are several ways to troubleshoot. + +One option is to build a custom docker image that includes your preferred debugging tools. Here is an example of adding +some tools to an existing docker image. + +```Dockerfile +FROM apachepulsar/pulsar:2.10.0 + +# Switch to root user to download tools +USER 0 + +# Install your preferred utilities +RUN apt-get update \ + && apt-get install -y vim net-tools unzip \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* + +# Assuming you still want to run as a non root user by default +USER 10000 +``` + +The remaining debug options depend on your environment. For example, if you have access to the host running your +container, you might be able to use the `docker exec` command to shell into the container. By using the `--user` +argument, you can run as the root user. + +If you're running your container on kubernetes, you can override the container's default user by setting the pod's +`securityContext`. + +Bitnami provides a helpful guide here: https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html. \ No newline at end of file diff --git a/docker/pulsar/Dockerfile b/docker/pulsar/Dockerfile index ae0c399d26a0e..1a19220d45575 100644 --- a/docker/pulsar/Dockerfile +++ b/docker/pulsar/Dockerfile @@ -33,7 +33,13 @@ COPY scripts/pulsar-zookeeper-ruok.sh /pulsar/bin COPY scripts/watch-znode.py /pulsar/bin COPY scripts/install-pulsar-client.sh /pulsar/bin -RUN mkdir /pulsar/data +# In order to support running this docker image as a container on OpenShift +# the final image needs to give the root group sufficient permission. +# The file permissions are preserved when copying files from this builder image to the target image. +RUN chmod -R g+w /pulsar/conf +# Presto writes logs to this directory (at least during tests), so we need to give the process permission +# to create those log directories. This should be removed when presto is removed. +RUN chmod g+w /pulsar/lib/presto ### Create 2nd stage from Ubuntu image ### and add OpenJDK and Python dependencies (for Pulsar functions) @@ -57,6 +63,14 @@ RUN sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirr RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10 +# Pulsar currently writes to the below directories, assuming the default configuration. +# Note that number 4 is the reason that pulsar components need write access to the /pulsar directory. +# 1. /pulsar/data - both bookkeepers and zookeepers use this directory +# 2. /pulsar/logs - function workers write to this directory and pulsar-admin initializes this directory +# 3. /pulsar/download - functions write to this directory +# 4. /pulsar - hadoop writes to this directory +RUN mkdir /pulsar && chmod g+w /pulsar + ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64 RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security ADD target/python-client/ /pulsar/pulsar-client @@ -67,4 +81,8 @@ ENV PULSAR_ROOT_LOGGER=INFO,CONSOLE COPY --from=pulsar /pulsar /pulsar WORKDIR /pulsar +# This script is intentionally run as the root user to make the dependencies available for all UIDs. RUN /pulsar/bin/install-pulsar-client.sh + +# The UID must be non-zero. Otherwise, it is arbitrary. No logic should rely on its specific value. +USER 10000 diff --git a/site2/docs/getting-started-docker.md b/site2/docs/getting-started-docker.md index 148995140ecf3..0d8fb11b169bd 100644 --- a/site2/docs/getting-started-docker.md +++ b/site2/docs/getting-started-docker.md @@ -20,6 +20,7 @@ A few things to note about this command: * The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every time the container is restarted. For details on the volumes you can use `docker volume inspect ` * For Docker on Windows make sure to configure it to use Linux containers + * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0). If you start Pulsar successfully, you will see `INFO`-level log messages like this: diff --git a/tests/docker-images/latest-version-image/Dockerfile b/tests/docker-images/latest-version-image/Dockerfile index 7300daad721b5..06cc18462b82d 100644 --- a/tests/docker-images/latest-version-image/Dockerfile +++ b/tests/docker-images/latest-version-image/Dockerfile @@ -21,6 +21,7 @@ FROM apachepulsar/pulsar:latest as pulsar-function-go +# Use root for builder USER root RUN rm -rf /var/lib/apt/lists/* && apt-get update @@ -55,6 +56,16 @@ FROM apachepulsar/pulsar-all:latest as pulsar-all ######################################## FROM apachepulsar/pulsar:latest +# Switch to run as the root user to simplify building container and then running +# supervisord. Each of the pulsar components are spawned by supervisord and their +# process configuration files specify that the process will be run with UID 10000. +# However, any processes exec'ing into the containers will run as root, by default. +USER root + +# We need to define the user in order for supervisord to work correctly +# We don't need a user defined in the public docker image, though. +RUN adduser -u 10000 --gid 0 --disabled-login --disabled-password --gecos '' pulsar + RUN rm -rf /var/lib/apt/lists/* && apt update RUN apt-get clean && apt-get update && apt-get install -y supervisor vim procps curl diff --git a/tests/docker-images/latest-version-image/conf/bookie.conf b/tests/docker-images/latest-version-image/conf/bookie.conf index f65a37c823795..97d2c72074831 100644 --- a/tests/docker-images/latest-version-image/conf/bookie.conf +++ b/tests/docker-images/latest-version-image/conf/bookie.conf @@ -24,3 +24,4 @@ stdout_logfile=/var/log/pulsar/bookie.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M -XX:MaxDirectMemorySize=512M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar bookie +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/broker.conf b/tests/docker-images/latest-version-image/conf/broker.conf index f8bf8e761398f..966c2545677b6 100644 --- a/tests/docker-images/latest-version-image/conf/broker.conf +++ b/tests/docker-images/latest-version-image/conf/broker.conf @@ -24,4 +24,4 @@ stdout_logfile=/var/log/pulsar/broker.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar broker - +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/functions_worker.conf b/tests/docker-images/latest-version-image/conf/functions_worker.conf index 3610b03251884..9b2890c8a2df0 100644 --- a/tests/docker-images/latest-version-image/conf/functions_worker.conf +++ b/tests/docker-images/latest-version-image/conf/functions_worker.conf @@ -24,4 +24,4 @@ stdout_logfile=/var/log/pulsar/functions_worker.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar functions-worker - +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/global-zk.conf b/tests/docker-images/latest-version-image/conf/global-zk.conf index bf56c5b62882a..dcc7701ce8c43 100644 --- a/tests/docker-images/latest-version-image/conf/global-zk.conf +++ b/tests/docker-images/latest-version-image/conf/global-zk.conf @@ -24,4 +24,4 @@ stdout_logfile=/var/log/pulsar/global-zk.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar configuration-store - +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/local-zk.conf b/tests/docker-images/latest-version-image/conf/local-zk.conf index 57681937791be..50ba294074d97 100644 --- a/tests/docker-images/latest-version-image/conf/local-zk.conf +++ b/tests/docker-images/latest-version-image/conf/local-zk.conf @@ -24,4 +24,4 @@ stdout_logfile=/var/log/pulsar/local-zk.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar zookeeper - +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/presto_worker.conf b/tests/docker-images/latest-version-image/conf/presto_worker.conf index 28e3c36661e09..3ddb11278b6b8 100644 --- a/tests/docker-images/latest-version-image/conf/presto_worker.conf +++ b/tests/docker-images/latest-version-image/conf/presto_worker.conf @@ -23,4 +23,5 @@ redirect_stderr=true stdout_logfile=/var/log/pulsar/presto_worker.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" -command=/pulsar/bin/pulsar sql-worker start \ No newline at end of file +command=/pulsar/bin/pulsar sql-worker start +user=pulsar diff --git a/tests/docker-images/latest-version-image/conf/proxy.conf b/tests/docker-images/latest-version-image/conf/proxy.conf index 8bc1a53fe4fa8..9e14a0e934bed 100644 --- a/tests/docker-images/latest-version-image/conf/proxy.conf +++ b/tests/docker-images/latest-version-image/conf/proxy.conf @@ -24,4 +24,4 @@ stdout_logfile=/var/log/pulsar/proxy.log directory=/pulsar environment=PULSAR_MEM="-Xmx128M",PULSAR_GC="-XX:+UseG1GC" command=/pulsar/bin/pulsar proxy - +user=pulsar From cfa733696e08c6db8a36d7df6c101d01d51552fc Mon Sep 17 00:00:00 2001 From: penghui Date: Mon, 21 Feb 2022 15:55:00 +0800 Subject: [PATCH 02/20] Added 2.10.0 release note --- site2/website/release-notes.md | 293 +++++++++++++++++++++++++++++++++ 1 file changed, 293 insertions(+) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 849294e82066b..40b2e8603fb81 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -1,6 +1,299 @@ ## Apache Pulsar Release Notes +### 2.10.0 +#### 2022-02-28 + +### Update notice +Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) +Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) +Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) +Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) +[PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) +[PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) +[PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +[PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) +Fixed netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) + +### PIPs +[PIP 79] Add lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) +[PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) +[PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) +[PIP 97] Update Authentication Interfaces to Include Async Authentication Methods [#12104](https://github.com/apache/pulsar/pull/12104) +[PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) +[PIP 104] Add new consumer type: TableView [#12838](https://github.com/apache/pulsar/pull/12838) +[PIP-105] Support pluggable entry filter in Dispatcher [#12869](https://github.com/apache/pulsar/pull/12869) [#12970](https://github.com/apache/pulsar/pull/12970) [#12979](https://github.com/apache/pulsar/pull/12979) +[PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility [#12536](https://github.com/apache/pulsar/pull/12536) +[PIP 107] Introduce chunk message ID [#12403](https://github.com/apache/pulsar/pull/12403) +[PIP 110] Support Topic metadata - PART-1 create topic with properties [#12818](https://github.com/apache/pulsar/pull/12818) +[PIP 121] Pulsar cluster level auto failover on client side [#13316](https://github.com/apache/pulsar/pull/13316) +[PIP-124] Create init subscription before sending message to DLQ [#13355](https://github.com/apache/pulsar/pull/13355) +[PIP-130] Apply redelivery backoff policy for ack timeout [#13707](https://github.com/apache/pulsar/pull/13707) +[PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set [#13599](https://github.com/apache/pulsar/pull/13599) +[PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) + +### Broker +[PIP 45] Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) +[PIP 45] Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) +[PIP 45] Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) +[PIP 45] Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) +[PIP 45] Removed old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) +[PIP 45] Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) +[PIP 45] Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) +[PIP 45] Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) +[PIP 45] Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) +[PIP 45] Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) +[PIP 45] Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) +[PIP 45] Allow configuring configurationMetadataStore when initialize cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) +[PIP 45] Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) +[PIP 45] Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) +[PIP 45] Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) +[PIP 45] Fixed metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) +Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) +Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) +Adding Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) +Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) +Support split the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) +Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) +Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) +Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) +Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) +Added uniform load shedder strategy to distribute traffic uniformly across brokers [#12902](https://github.com/apache/pulsar/pull/12902) +Provide option to split bundle based on load [#12378](https://github.com/apache/pulsar/pull/12378) +Use AuthorizationService#grantPermissionAsync to grant topic permission [#12515](https://github.com/apache/pulsar/pull/12515) +Only store authentication data after authentication is complete [#12077](https://github.com/apache/pulsar/pull/12077) +Allow to have different instances LocalMemoryMetadataStore that share the same state [#12390](https://github.com/apache/pulsar/pull/12390) +Allow `GetTopicsOfNamespace` op with `consume` permission [#12600](https://github.com/apache/pulsar/pull/12600) +Added local filesystem backend for package manager [#12708](https://github.com/apache/pulsar/pull/12708) +Added stop replicator producer logic when start replicator cluster failed [#12724](https://github.com/apache/pulsar/pull/12724) +Apply PolicyHierarchyValue to inactiveTopicPolicies [#12687](https://github.com/apache/pulsar/pull/12687) +Fixed can not get leader broker in follower brokers [#11353](https://github.com/apache/pulsar/pull/11353) +Fixed broker gc log options [#11285](https://github.com/apache/pulsar/pull/11285) +Fixed NPE of ZkBookieRackAffinityMapping [#11947](https://github.com/apache/pulsar/pull/11947) +Fixed prefix setting in JWT authn and avoid multi calls for the getProperty [#12132](https://github.com/apache/pulsar/pull/12132) +Fixed missed check exit code of stop before calling start [#12368](https://github.com/apache/pulsar/pull/12368) +Fixed getting children of parent nodes in LocalMemoryMetadataStore [#12491](https://github.com/apache/pulsar/pull/12491) +Fixed collection get bug in ResourceGroupService [#12499](https://github.com/apache/pulsar/pull/12499) +Fixed deadlock in metadata-store callback thread [#12753](https://github.com/apache/pulsar/pull/12753) +Improve exceptions thrown when handling the schema resource [#12155](https://github.com/apache/pulsar/pull/12155) +Trim topic name [#12453](https://github.com/apache/pulsar/pull/12453) +Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic [#12658](https://github.com/apache/pulsar/pull/12658) +Optimized isValidPath check in MetadataStore [#12663](https://github.com/apache/pulsar/pull/12663) +Close connection after receiving unexpected SendCommand [#12780](https://github.com/apache/pulsar/pull/12780) +Ensure cache is refreshed (and not just invalidated) after a store write [#12788](https://github.com/apache/pulsar/pull/12788) +Optimized topicMaxMessageSize with topic local cache. [#12830](https://github.com/apache/pulsar/pull/12830) +Optimized blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer [#12874](https://github.com/apache/pulsar/pull/12874) +Only refresh metadata if path is already in cache after write. [#12896](https://github.com/apache/pulsar/pull/12896) +Optimized put and get methods in AbstractMetadataStore [#12916](https://github.com/apache/pulsar/pull/12916) +Fixed zk-node leak of admin path [#12972](https://github.com/apache/pulsar/pull/12972) +Optimized TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies [#13000](https://github.com/apache/pulsar/pull/13000) +Fixed race condition in FaultInjectionMetadataStore#programmedFailure [#13007](https://github.com/apache/pulsar/pull/13007) +Fixed usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts [#13025](https://github.com/apache/pulsar/pull/13025) +Consider topics in pulsar/system namespace as system topics [#13050](https://github.com/apache/pulsar/pull/13050) +Fixed wrong result for looking up a non-exist topic by rest api [#13055](https://github.com/apache/pulsar/pull/13055) +Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes [#13063](https://github.com/apache/pulsar/pull/13063) +Make load-balancer config dynamic for the runtime tuning [#13074](https://github.com/apache/pulsar/pull/13074) +Optimized TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https://github.com/apache/pulsar/pull/13082) +Load balancer supports disabling max-session for bundle split [#13108](https://github.com/apache/pulsar/pull/13108) +Optimized TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies [#13121](https://github.com/apache/pulsar/pull/13121) +Log thread dump when Zookeeper session expires to help detect possible deadlocks [#13124](https://github.com/apache/pulsar/pull/13124) +Add removeMaxConsumersPerSubscription method for v1 namespace [#13192](https://github.com/apache/pulsar/pull/13192) +Fixed error value for 1MB in metrics [#13213](https://github.com/apache/pulsar/pull/13213) +Optimized TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies [#13241](https://github.com/apache/pulsar/pull/13241) +Added schema compatibility strategy on topic level [#13297](https://github.com/apache/pulsar/pull/13297) +Optimized TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies [#13361](https://github.com/apache/pulsar/pull/13361) +Optimized TopicPolicies#replicationClusters with HierarchyTopicPolicies [#13495](https://github.com/apache/pulsar/pull/13495) +Fixed compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) +Fixed race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) +Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) +Return null instead of RestException when get bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) +Fixed failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) +Optimized TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) +Fixed raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) +Optimized TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies [#13618](https://github.com/apache/pulsar/pull/13618) +Optimized TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies [#13649](https://github.com/apache/pulsar/pull/13649) +Optimized TopicPolicies#compactionThreshold with HierarchyTopicPolicies [#13710](https://github.com/apache/pulsar/pull/13710) +Added deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) +Fixed NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) +Optimized deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) +Fixed call sync method in async rest api for `internalDeletePartitionedTopic` [#13805](https://github.com/apache/pulsar/pull/13805) +Use shared executors for broker and geo-replication clients [#13839](https://github.com/apache/pulsar/pull/13839) +Fixed call sync method in async rest api for internalCreateSubscription. [#13873](https://github.com/apache/pulsar/pull/13873) +Fixed call sync method in async rest api for internalGetBacklogSizeByMessageId [#13871](https://github.com/apache/pulsar/pull/13871) +Fixed call sync method in async rest api for internalTriggerCompaction [#13853](https://github.com/apache/pulsar/pull/13853) +Fixed call sync method in async rest api for internalGetManagedLedgerInfo [#13847](https://github.com/apache/pulsar/pull/13847) +Fixed call sync method in async rest api for internalGetSubscriptions [#13846](https://github.com/apache/pulsar/pull/13846) +Fixed call sync method in async rest api for internalUnloadTopic [#13845](https://github.com/apache/pulsar/pull/13845) +Fixed call sync method in async rest api for internalGetLastMessageId [#13882](https://github.com/apache/pulsar/pull/13882) +Fixed call sync method in async rest api for internalExpireMessagesByTimestamp [#13880](https://github.com/apache/pulsar/pull/13880) +Fixed call sync method in async rest api for internalResetCursorOnPosition [#13879](https://github.com/apache/pulsar/pull/13879) +Fixed call sync method in async rest api for internalExpireMessagesByPosition [#13878](https://github.com/apache/pulsar/pull/13878) +Fixed call sync method in async rest api for internalGetMessageById. [#13876](https://github.com/apache/pulsar/pull/13876) +Fixed call sync method in async rest api for internalSetReplicatedSubscriptionStatus [#13887](https://github.com/apache/pulsar/pull/13887) +Fixed call sync method in async rest api for internalGetPartitionedStats [#13886](https://github.com/apache/pulsar/pull/13886) +Fixed call sync method in onPoliciesUpdate method [#13885](https://github.com/apache/pulsar/pull/13885) +Fixed call sync method in async rest api for internalGetPartitionedStatsInternal [#13884](https://github.com/apache/pulsar/pull/13884) +Fixed call sync method in async rest api for internalTerminatePartitionedTopic [#13890](https://github.com/apache/pulsar/pull/13890) +Make PulsarAuthorizationProvider#grantPermissionAsync actually async [#13897](https://github.com/apache/pulsar/pull/13897) +Fixed no value present when updating dispatch rate [#13900](https://github.com/apache/pulsar/pull/13900) +Fixed call sync method in async rest api for internalSkipAllMessages [#13901](https://github.com/apache/pulsar/pull/13901) +Optimized retentionPolicies with HierarchyTopicPolicies [#13909](https://github.com/apache/pulsar/pull/13909) +Improved 403 response message wih consume permission on namespace [#13912](https://github.com/apache/pulsar/pull/13912) +Fixed arithmetic exception for uniform load shedder [#13914](https://github.com/apache/pulsar/pull/13914) +Added threshold for each unload round for uniform load shedder [#13915](https://github.com/apache/pulsar/pull/13915) +Fixed call sync method in async rest api for internalDeleteSubscription #13884 [#13917](https://github.com/apache/pulsar/pull/13917) +Change ``BrokersBase`` api ``getActiveBrokers`` and ``getLeaderBroker`` to pure async. [#13935](https://github.com/apache/pulsar/pull/13935) +Do not create missing topic when loading namespace [#13948](https://github.com/apache/pulsar/pull/13948) +Only reply to client when code completes producerFuture [#13949](https://github.com/apache/pulsar/pull/13949) +Improve error logging for topic not found [#13950](https://github.com/apache/pulsar/pull/13950) +Fixed call sync method in async rest api for preValidation. [#13962](https://github.com/apache/pulsar/pull/13962) +Fixed call sync method in async rest api for internalSetReplicationClusters and internalRemoveReplicationClusters [#13961](https://github.com/apache/pulsar/pull/13961) +Make validateTenantOperation method async in PulsarWebResource [#14008](https://github.com/apache/pulsar/pull/14008) +Optimized maxUnackedMessagesOnSubscription with HierarchyTopicPolicies [#14011](https://github.com/apache/pulsar/pull/14011) +Set default value of applied to false in getSchemaCompatibilityStrategy [#14012](https://github.com/apache/pulsar/pull/14012) +Make validateTopicPolicyOperation method async in PulsarWebResource [#14024](https://github.com/apache/pulsar/pull/14024) +Fixed producerFuture.completeExceptionally not called before sendErrorResponse [#14025](https://github.com/apache/pulsar/pull/14025) +Make triggerOffload method async [#14027](https://github.com/apache/pulsar/pull/14027) +Make offloadStatus method async [#14029](https://github.com/apache/pulsar/pull/14029) +Make validateNamespacePolicyOperation method async in PulsarWebResource [#14033](https://github.com/apache/pulsar/pull/14033) +Make internalSkipMessages method async [#14045](https://github.com/apache/pulsar/pull/14045) +Make PersistentTopicsBase#internalSetBacklogQuota async [#14051](https://github.com/apache/pulsar/pull/14051) +Adjusted the validation for policy schemaCompatibilityStrategy [#14061](https://github.com/apache/pulsar/pull/14061) +Remove Persistent Topics v3 API - use custom media type instead [#14117](https://github.com/apache/pulsar/pull/14117) +Fix race condition in PulsarLedgerIdGenerator#generateShortLedgerId [#14118](https://github.com/apache/pulsar/pull/14118) +Optimized topic policy with HierarchyTopicPolicies about subscriptionDispatchRate [#14151](https://github.com/apache/pulsar/pull/14151) +Make `BrokerBase#deleteDynamicConfiguration` to pure async method [#14163](https://github.com/apache/pulsar/pull/14163) +Set default value of applied to false on topic policy [#14181](https://github.com/apache/pulsar/pull/14181) +Change broker producer fence log level [#14196](https://github.com/apache/pulsar/pull/14196) +Removed duplicated filter for UniformLoadShedder#findBundlesForUnloading [#14198](https://github.com/apache/pulsar/pull/14198) +Adjusted topic exists check logic in http lookup process [#14199](https://github.com/apache/pulsar/pull/14199) +Fixed NPE of internalExpireMessagesByTimestamp [#14243](https://github.com/apache/pulsar/pull/14243) +Fixed rackaware placement policy does not take effect after delete rack configuration [#14248](https://github.com/apache/pulsar/pull/14248) +Fixed print error log when server return redirect (http code 307) [#14259](https://github.com/apache/pulsar/pull/14259) +Optimized topic policy with HierarchyTopicPolicies about publishRate [#14267](https://github.com/apache/pulsar/pull/14267) +Fixed ack-hole and backlog for persistent-replicator [#14282](https://github.com/apache/pulsar/pull/14282) +Fixed NPE in internalSkipMessages [#14297](https://github.com/apache/pulsar/pull/14297) +Validate blank advertised listener name [#14306](https://github.com/apache/pulsar/pull/14306) +Let entries expire in the metadata caches [#14154](https://github.com/apache/pulsar/pull/14154) +Fixed transaction system topic loop creation [#12749](https://github.com/apache/pulsar/pull/12749) +Fixed topic transaction buffer handle null snapshot [#12758](https://github.com/apache/pulsar/pull/12758) +Optimized changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) + +### Clients +[Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) +[Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) +[Java] Fixed the producer OOM if got exception while add message to batch container [#12170](https://github.com/apache/pulsar/pull/12170) +[Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) +[Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) +[Java] Fixed confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) +[Java] Fixed parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) +[Java] Added getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) +[Java] Allowed config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) +[Java] Support add custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) +[Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) +[Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) +[Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) +[Java] Log producer batchSize and msgSize percentiles [#14229](https://github.com/apache/pulsar/pull/14229) +[C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) +[C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) +[C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) +[C++] Support more pulsar command name and version in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) +[C++] Optimize MessageBuilder and SharedBuffer to avoid unnecessary memory copy [#13293](https://github.com/apache/pulsar/pull/13293) +[C++] Fixed in Apple Silicon macOS the clang-format cannot find [#13333](https://github.com/apache/pulsar/pull/13333) +[C++] Added clang-format check for Pulsar Wireshark dissector [#13349](https://github.com/apache/pulsar/pull/13349) +[C++] Adjust clang-format search names [#13369](https://github.com/apache/pulsar/pull/13369) +[C++] Fixed Wireshark dissector decode send command metadata behavior [#13471](https://github.com/apache/pulsar/pull/13471) +[C++] PIP 37: Support large message size [#13627](https://github.com/apache/pulsar/pull/13627) +[C++] Fix the consumer configuration inconsistency with Java client [#14070](https://github.com/apache/pulsar/pull/14070) +[Python] Provide __str__ operator for BytesSchema [#12593](https://github.com/apache/pulsar/pull/12593) +[Python] Support is_connected in Python Client [#13662](https://github.com/apache/pulsar/pull/13662) +[C] Add missing includes in reader_configuration.h [#12966](https://github.com/apache/pulsar/pull/12966) +[C] Add pulsar_client_subscribe_multi_topics and pulsar_client_subscribe_pattern [#12965](https://github.com/apache/pulsar/pull/12965) + +### Pulsar IO and Pulsar Functions + +[Functions] Prevent NPE while stopping a non started Pulsar LogAppender [#12643](https://github.com/apache/pulsar/pull/12643) +[Functions] Allow configuring different implementations for Pulsar functions state store [#12646](https://github.com/apache/pulsar/pull/12646) +[Functions] Clean os even when statusFuture complete exceptionally [#12767](https://github.com/apache/pulsar/pull/12767) +[Functions] Override inactive_topic_policies in Pulsar Functions namespace creation [#13048](https://github.com/apache/pulsar/pull/13048) +[Functions] Added possibility to pass additional JVM arguments to the function JVM (additionalJavaRuntimeArguments) [#13282](https://github.com/apache/pulsar/pull/13282) +[Functions] Fixed `getTlsTrustChainBytes` not work when functions worker not run with broker [#13875](https://github.com/apache/pulsar/pull/13875) +[Functions] Fixed distributed log metadata not correctly initialized [#13891](https://github.com/apache/pulsar/pull/13891) +[Functions] Removed sensitive information from log [#14159](https://github.com/apache/pulsar/pull/14159) +[IO Connector] Pass client builder if no service url provided to debezium connector [#12145](https://github.com/apache/pulsar/pull/12145) +[IO Connector] Fixed: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test [12374](https://github.com/apache/pulsar/pull/12374) +[IO Connector] Fixed the marshal and unmarshal the sink config [#12625](https://github.com/apache/pulsar/pull/12625) +[IO Connector] Support rename the file has been processed [#13373](https://github.com/apache/pulsar/pull/13373) +[IO Connector] Support Fixed and ENUM datatypes for ElasticSearch Sink [#13800](https://github.com/apache/pulsar/pull/13800) +[IO Connector] Implement --retain-key-ordering (KEY_SHARED subscription) for Sinks [#14083](https://github.com/apache/pulsar/pull/14083) + +### Observability + +[Broker] Exposed broker bundles metrics to prometheus [#12366](https://github.com/apache/pulsar/pull/12366) +[Broker] Added publishRateLimitedTimes to topic metrics [#13538](https://github.com/apache/pulsar/pull/13538) +[Broker] Fixed bundle metrics would overwrite loadbalance metrics [#13641](https://github.com/apache/pulsar/pull/13641) +[Broker] Fixed managed cursor acknowledgment state metric names [#13844](https://github.com/apache/pulsar/pull/13844) +[Java Client] Added pending-queue size metrics to producer stats [#12674](https://github.com/apache/pulsar/pull/12674) + +### CLI +[Pulsar Admin] Support get a mapping of brokers to partitioned topics [#11763](https://github.com/apache/pulsar/pull/11763) +[Pulsar Admin] Support terminate a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) +[Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) +[Pulsar Admin] Added remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) +[Pulsar Admin] Fixed output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) +[Pulsar Admin] Added a metric to get the earliest time in the backlog [#12523](https://github.com/apache/pulsar/pull/12523) +[Pulsar Admin] Reduced severity of log "refreshing key manager" in KeyManagerProxy [#12594](https://github.com/apache/pulsar/pull/12594) +[Pulsar Admin] Support to get list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) +[Pulsar Admin] Added get-replicated-subscription-status command for topic [#12891](https://github.com/apache/pulsar/pull/12891) +[Pulsar Admin] Added remove-subscription-types-enabled command for topic [#12983](https://github.com/apache/pulsar/pull/12983) +[Pulsar Admin] Validate the size options in cmd for topic and namespace [#13002](https://github.com/apache/pulsar/pull/13002) +[Pulsar Admin] Support subscription across multiple clusters [#13482](https://github.com/apache/pulsar/pull/13482) +[Pulsar Admin] Support deduplication across multiple clusters [#13487](https://github.com/apache/pulsar/pull/13487) +[Pulsar Admin] Support message TTL across multiple clusters [#13484](https://github.com/apache/pulsar/pull/13484) +[Pulsar Admin] Support persistence policies across multiple clusters [#13483](https://github.com/apache/pulsar/pull/13483) +[Pulsar Admin] Support publish rate across multiple clusters [#13496](https://github.com/apache/pulsar/pull/13496) +[Pulsar Admin] Support max consumers across multiple clusters [#13521](https://github.com/apache/pulsar/pull/13521) +[Pulsar Admin] Support max producer across multiple clusters [#13519](https://github.com/apache/pulsar/pull/13519) +[Pulsar Admin] Support compaction threshold across multiple clusters [#13513](https://github.com/apache/pulsar/pull/13513) +[Pulsar Admin] Support offload policies across multiple clusters [#13534](https://github.com/apache/pulsar/pull/13534) +[Pulsar Admin] Support max unacked messages per consumer across multiple clusters [#13547](https://github.com/apache/pulsar/pull/13547) +[Pulsar Admin] Support max message size support across multiple clusters [#13579](https://github.com/apache/pulsar/pull/13579) +[Pulsar Admin] Support deduplication snapshot interval across multiple clusters [#13578](https://github.com/apache/pulsar/pull/13578) +[Pulsar Admin] Support delayed delivery policy across multiple clusters [#13550](https://github.com/apache/pulsar/pull/13550) +[Pulsar Admin] Support max unacked messages on subscription across multiple clusters [#13549](https://github.com/apache/pulsar/pull/13549) +[Pulsar Admin] Support replicator dispatch rate across multiple clusters [#13624](https://github.com/apache/pulsar/pull/13624) +[Pulsar Admin] Support max subscriptions per topic across multiple clusters [#13623](https://github.com/apache/pulsar/pull/13623) +[Pulsar Admin] Move schema compatibility strategy cmd from topics to topicPolicies [#14225](https://github.com/apache/pulsar/pull/14225) +[Pulsar CLI] Added restart command to pulsar-daemon [#12279](https://github.com/apache/pulsar/pull/12279) +[Pulsar CLI] Print log for standalone when configuration is failed to load [#12280](https://github.com/apache/pulsar/pull/12280) +[Client Tool] Support Disabling Replication [#13659](https://github.com/apache/pulsar/pull/13659) +[Perf Tool] Added auth for transaction perf [#14271](https://github.com/apache/pulsar/pull/14271) + +### Others +[Proxy] Fixed Pulsar Proxy to re-use authentication instance [#12245](https://github.com/apache/pulsar/pull/12245) +[Proxy] Fixed auto-cert refresh when proxy connects to broker [#14130](https://github.com/apache/pulsar/pull/14130) +[WebSocket] Fixed the batch message ack [#12530](https://github.com/apache/pulsar/pull/12530) +[Package Management] check service status before run commands [#12847](https://github.com/apache/pulsar/pull/12847) +[Package Management] Add package management filesystem storage into the distribution lib [#13202](https://github.com/apache/pulsar/pull/13202) +[Package Management] Fix the filesystem storage failure [#13218](https://github.com/apache/pulsar/pull/13218) +[Schema] Fix pulsar use json or avro primitive schema [#12886](https://github.com/apache/pulsar/pull/12886) +[Tiered Storage] Fixed FileSystemManagedLedgerOffloader can not cleanup outdated ledger [#12309](https://github.com/apache/pulsar/pull/12309) +[Tiered Storage] Fixed NoClassDefFoundError: com/google/inject/AbstractModule in pulsar-io/batch-data-generator and Jcloud offloader [#14150](https://github.com/apache/pulsar/pull/14150) +Added lua wireshark for pulsar [#13564](https://github.com/apache/pulsar/pull/13564) +Fixed missing new line at the end of proxy.conf [#14359](https://github.com/apache/pulsar/pull/14359) + +### Library updates + +Upgraded function's Go client version to v0.7.0 [#12839](https://github.com/apache/pulsar/pull/12839) +Upgraded dependencies (guava and what brought in older guava) to get rid of the guava-related CVE-2018-10237 and CVE-2020-8908 [#13716](https://github.com/apache/pulsar/pull/13716) +Use dependencyManagement to enforce snakeyaml version to 1.30 [#13722](https://github.com/apache/pulsar/pull/13722) +Upgraded dependencies to get rid of pulsar-io/jdbc related CVE-2020-13692 [#13753](https://github.com/apache/pulsar/pull/13753) +Removed --illegal-access errors resulting from Google Guice (upgrade to 5.0.1 and JClouds to 2.4.0) [#13810](https://github.com/apache/pulsar/pull/13810) +Upgraded jakarta.el to 3.0.4 to get rid of CVE-2021-28170 [#13943](https://github.com/apache/pulsar/pull/13943) +Upgraded Netty to 4.1.73.Final [#13981](https://github.com/apache/pulsar/pull/13981) +Removed net.jodah.failsafe dependency (fix JDK17 build) [#14124](https://github.com/apache/pulsar/pull/14124) +Upgraded netty version to 4.1.74.Final [#14257](https://github.com/apache/pulsar/pull/14257) + ### 2.9.1 #### 2021-12-20 From c4e1520e6de7fe2cc8e71f4a93a07816e7b90cc7 Mon Sep 17 00:00:00 2001 From: penghui Date: Mon, 21 Feb 2022 16:07:08 +0800 Subject: [PATCH 03/20] Add #13376 to update notice section --- site2/website/release-notes.md | 1 + 1 file changed, 1 insertion(+) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 40b2e8603fb81..cd59c4d0158e0 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -12,6 +12,7 @@ Align configurations defaults between default file and Java object (broker.conf, [PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) [PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) [PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) Fixed netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) From 9c4c1b34e5522bf3542a8863f2d80b6759afd549 Mon Sep 17 00:00:00 2001 From: penghui Date: Tue, 22 Feb 2022 10:15:57 +0800 Subject: [PATCH 04/20] Address comments --- site2/website/release-notes.md | 542 ++++++++++++++++----------------- 1 file changed, 271 insertions(+), 271 deletions(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index cd59c4d0158e0..cbc80370e8562 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -4,296 +4,296 @@ ### 2.10.0 #### 2022-02-28 -### Update notice -Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) -Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) -Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) -Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) -[PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) -[PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) -[PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) -Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) -[PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) -Fixed netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) +### Important notice +- Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) +- Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) +- Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) +- Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) +- [PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) +- [PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) +- [PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +- Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) +- [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) +- Fix netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) ### PIPs -[PIP 79] Add lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) -[PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) -[PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) -[PIP 97] Update Authentication Interfaces to Include Async Authentication Methods [#12104](https://github.com/apache/pulsar/pull/12104) -[PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) -[PIP 104] Add new consumer type: TableView [#12838](https://github.com/apache/pulsar/pull/12838) -[PIP-105] Support pluggable entry filter in Dispatcher [#12869](https://github.com/apache/pulsar/pull/12869) [#12970](https://github.com/apache/pulsar/pull/12970) [#12979](https://github.com/apache/pulsar/pull/12979) -[PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility [#12536](https://github.com/apache/pulsar/pull/12536) -[PIP 107] Introduce chunk message ID [#12403](https://github.com/apache/pulsar/pull/12403) -[PIP 110] Support Topic metadata - PART-1 create topic with properties [#12818](https://github.com/apache/pulsar/pull/12818) -[PIP 121] Pulsar cluster level auto failover on client side [#13316](https://github.com/apache/pulsar/pull/13316) -[PIP-124] Create init subscription before sending message to DLQ [#13355](https://github.com/apache/pulsar/pull/13355) -[PIP-130] Apply redelivery backoff policy for ack timeout [#13707](https://github.com/apache/pulsar/pull/13707) -[PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set [#13599](https://github.com/apache/pulsar/pull/13599) -[PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) +- [PIP 79] Added lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) +- [PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) +- [PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) +- [PIP 97] Update Authentication Interfaces to Include Async Authentication Methods [#12104](https://github.com/apache/pulsar/pull/12104) +- [PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) +- [PIP 104] Add new consumer type: TableView [#12838](https://github.com/apache/pulsar/pull/12838) +- [PIP-105] Support pluggable entry filter in Dispatcher [#12869](https://github.com/apache/pulsar/pull/12869) [#12970](https://github.com/apache/pulsar/pull/12970) [#12979](https://github.com/apache/pulsar/pull/12979) +- [PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility [#12536](https://github.com/apache/pulsar/pull/12536) +- [PIP 107] Introduce chunk message ID [#12403](https://github.com/apache/pulsar/pull/12403) +- [PIP 110] Support Topic metadata - PART-1 create topic with properties [#12818](https://github.com/apache/pulsar/pull/12818) +- [PIP 121] Pulsar cluster level auto failover on client side [#13316](https://github.com/apache/pulsar/pull/13316) +- [PIP-124] Create init subscription before sending message to DLQ [#13355](https://github.com/apache/pulsar/pull/13355) +- [PIP-130] Apply redelivery backoff policy for ack timeout [#13707](https://github.com/apache/pulsar/pull/13707) +- [PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set [#13599](https://github.com/apache/pulsar/pull/13599) +- [PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) ### Broker -[PIP 45] Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) -[PIP 45] Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) -[PIP 45] Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) -[PIP 45] Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) -[PIP 45] Removed old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) -[PIP 45] Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) -[PIP 45] Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) -[PIP 45] Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) -[PIP 45] Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) -[PIP 45] Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) -[PIP 45] Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) -[PIP 45] Allow configuring configurationMetadataStore when initialize cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) -[PIP 45] Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) -[PIP 45] Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) -[PIP 45] Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) -[PIP 45] Fixed metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) -Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) -Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) -Adding Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) -Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) -Support split the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) -Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) -Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) -Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) -Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) -Added uniform load shedder strategy to distribute traffic uniformly across brokers [#12902](https://github.com/apache/pulsar/pull/12902) -Provide option to split bundle based on load [#12378](https://github.com/apache/pulsar/pull/12378) -Use AuthorizationService#grantPermissionAsync to grant topic permission [#12515](https://github.com/apache/pulsar/pull/12515) -Only store authentication data after authentication is complete [#12077](https://github.com/apache/pulsar/pull/12077) -Allow to have different instances LocalMemoryMetadataStore that share the same state [#12390](https://github.com/apache/pulsar/pull/12390) -Allow `GetTopicsOfNamespace` op with `consume` permission [#12600](https://github.com/apache/pulsar/pull/12600) -Added local filesystem backend for package manager [#12708](https://github.com/apache/pulsar/pull/12708) -Added stop replicator producer logic when start replicator cluster failed [#12724](https://github.com/apache/pulsar/pull/12724) -Apply PolicyHierarchyValue to inactiveTopicPolicies [#12687](https://github.com/apache/pulsar/pull/12687) -Fixed can not get leader broker in follower brokers [#11353](https://github.com/apache/pulsar/pull/11353) -Fixed broker gc log options [#11285](https://github.com/apache/pulsar/pull/11285) -Fixed NPE of ZkBookieRackAffinityMapping [#11947](https://github.com/apache/pulsar/pull/11947) -Fixed prefix setting in JWT authn and avoid multi calls for the getProperty [#12132](https://github.com/apache/pulsar/pull/12132) -Fixed missed check exit code of stop before calling start [#12368](https://github.com/apache/pulsar/pull/12368) -Fixed getting children of parent nodes in LocalMemoryMetadataStore [#12491](https://github.com/apache/pulsar/pull/12491) -Fixed collection get bug in ResourceGroupService [#12499](https://github.com/apache/pulsar/pull/12499) -Fixed deadlock in metadata-store callback thread [#12753](https://github.com/apache/pulsar/pull/12753) -Improve exceptions thrown when handling the schema resource [#12155](https://github.com/apache/pulsar/pull/12155) -Trim topic name [#12453](https://github.com/apache/pulsar/pull/12453) -Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic [#12658](https://github.com/apache/pulsar/pull/12658) -Optimized isValidPath check in MetadataStore [#12663](https://github.com/apache/pulsar/pull/12663) -Close connection after receiving unexpected SendCommand [#12780](https://github.com/apache/pulsar/pull/12780) -Ensure cache is refreshed (and not just invalidated) after a store write [#12788](https://github.com/apache/pulsar/pull/12788) -Optimized topicMaxMessageSize with topic local cache. [#12830](https://github.com/apache/pulsar/pull/12830) -Optimized blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer [#12874](https://github.com/apache/pulsar/pull/12874) -Only refresh metadata if path is already in cache after write. [#12896](https://github.com/apache/pulsar/pull/12896) -Optimized put and get methods in AbstractMetadataStore [#12916](https://github.com/apache/pulsar/pull/12916) -Fixed zk-node leak of admin path [#12972](https://github.com/apache/pulsar/pull/12972) -Optimized TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies [#13000](https://github.com/apache/pulsar/pull/13000) -Fixed race condition in FaultInjectionMetadataStore#programmedFailure [#13007](https://github.com/apache/pulsar/pull/13007) -Fixed usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts [#13025](https://github.com/apache/pulsar/pull/13025) -Consider topics in pulsar/system namespace as system topics [#13050](https://github.com/apache/pulsar/pull/13050) -Fixed wrong result for looking up a non-exist topic by rest api [#13055](https://github.com/apache/pulsar/pull/13055) -Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes [#13063](https://github.com/apache/pulsar/pull/13063) -Make load-balancer config dynamic for the runtime tuning [#13074](https://github.com/apache/pulsar/pull/13074) -Optimized TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https://github.com/apache/pulsar/pull/13082) -Load balancer supports disabling max-session for bundle split [#13108](https://github.com/apache/pulsar/pull/13108) -Optimized TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies [#13121](https://github.com/apache/pulsar/pull/13121) -Log thread dump when Zookeeper session expires to help detect possible deadlocks [#13124](https://github.com/apache/pulsar/pull/13124) -Add removeMaxConsumersPerSubscription method for v1 namespace [#13192](https://github.com/apache/pulsar/pull/13192) -Fixed error value for 1MB in metrics [#13213](https://github.com/apache/pulsar/pull/13213) -Optimized TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies [#13241](https://github.com/apache/pulsar/pull/13241) -Added schema compatibility strategy on topic level [#13297](https://github.com/apache/pulsar/pull/13297) -Optimized TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies [#13361](https://github.com/apache/pulsar/pull/13361) -Optimized TopicPolicies#replicationClusters with HierarchyTopicPolicies [#13495](https://github.com/apache/pulsar/pull/13495) -Fixed compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) -Fixed race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) -Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) -Return null instead of RestException when get bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) -Fixed failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) -Optimized TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) -Fixed raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) -Optimized TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies [#13618](https://github.com/apache/pulsar/pull/13618) -Optimized TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies [#13649](https://github.com/apache/pulsar/pull/13649) -Optimized TopicPolicies#compactionThreshold with HierarchyTopicPolicies [#13710](https://github.com/apache/pulsar/pull/13710) -Added deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) -Fixed NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) -Optimized deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) -Fixed call sync method in async rest api for `internalDeletePartitionedTopic` [#13805](https://github.com/apache/pulsar/pull/13805) -Use shared executors for broker and geo-replication clients [#13839](https://github.com/apache/pulsar/pull/13839) -Fixed call sync method in async rest api for internalCreateSubscription. [#13873](https://github.com/apache/pulsar/pull/13873) -Fixed call sync method in async rest api for internalGetBacklogSizeByMessageId [#13871](https://github.com/apache/pulsar/pull/13871) -Fixed call sync method in async rest api for internalTriggerCompaction [#13853](https://github.com/apache/pulsar/pull/13853) -Fixed call sync method in async rest api for internalGetManagedLedgerInfo [#13847](https://github.com/apache/pulsar/pull/13847) -Fixed call sync method in async rest api for internalGetSubscriptions [#13846](https://github.com/apache/pulsar/pull/13846) -Fixed call sync method in async rest api for internalUnloadTopic [#13845](https://github.com/apache/pulsar/pull/13845) -Fixed call sync method in async rest api for internalGetLastMessageId [#13882](https://github.com/apache/pulsar/pull/13882) -Fixed call sync method in async rest api for internalExpireMessagesByTimestamp [#13880](https://github.com/apache/pulsar/pull/13880) -Fixed call sync method in async rest api for internalResetCursorOnPosition [#13879](https://github.com/apache/pulsar/pull/13879) -Fixed call sync method in async rest api for internalExpireMessagesByPosition [#13878](https://github.com/apache/pulsar/pull/13878) -Fixed call sync method in async rest api for internalGetMessageById. [#13876](https://github.com/apache/pulsar/pull/13876) -Fixed call sync method in async rest api for internalSetReplicatedSubscriptionStatus [#13887](https://github.com/apache/pulsar/pull/13887) -Fixed call sync method in async rest api for internalGetPartitionedStats [#13886](https://github.com/apache/pulsar/pull/13886) -Fixed call sync method in onPoliciesUpdate method [#13885](https://github.com/apache/pulsar/pull/13885) -Fixed call sync method in async rest api for internalGetPartitionedStatsInternal [#13884](https://github.com/apache/pulsar/pull/13884) -Fixed call sync method in async rest api for internalTerminatePartitionedTopic [#13890](https://github.com/apache/pulsar/pull/13890) -Make PulsarAuthorizationProvider#grantPermissionAsync actually async [#13897](https://github.com/apache/pulsar/pull/13897) -Fixed no value present when updating dispatch rate [#13900](https://github.com/apache/pulsar/pull/13900) -Fixed call sync method in async rest api for internalSkipAllMessages [#13901](https://github.com/apache/pulsar/pull/13901) -Optimized retentionPolicies with HierarchyTopicPolicies [#13909](https://github.com/apache/pulsar/pull/13909) -Improved 403 response message wih consume permission on namespace [#13912](https://github.com/apache/pulsar/pull/13912) -Fixed arithmetic exception for uniform load shedder [#13914](https://github.com/apache/pulsar/pull/13914) -Added threshold for each unload round for uniform load shedder [#13915](https://github.com/apache/pulsar/pull/13915) -Fixed call sync method in async rest api for internalDeleteSubscription #13884 [#13917](https://github.com/apache/pulsar/pull/13917) -Change ``BrokersBase`` api ``getActiveBrokers`` and ``getLeaderBroker`` to pure async. [#13935](https://github.com/apache/pulsar/pull/13935) -Do not create missing topic when loading namespace [#13948](https://github.com/apache/pulsar/pull/13948) -Only reply to client when code completes producerFuture [#13949](https://github.com/apache/pulsar/pull/13949) -Improve error logging for topic not found [#13950](https://github.com/apache/pulsar/pull/13950) -Fixed call sync method in async rest api for preValidation. [#13962](https://github.com/apache/pulsar/pull/13962) -Fixed call sync method in async rest api for internalSetReplicationClusters and internalRemoveReplicationClusters [#13961](https://github.com/apache/pulsar/pull/13961) -Make validateTenantOperation method async in PulsarWebResource [#14008](https://github.com/apache/pulsar/pull/14008) -Optimized maxUnackedMessagesOnSubscription with HierarchyTopicPolicies [#14011](https://github.com/apache/pulsar/pull/14011) -Set default value of applied to false in getSchemaCompatibilityStrategy [#14012](https://github.com/apache/pulsar/pull/14012) -Make validateTopicPolicyOperation method async in PulsarWebResource [#14024](https://github.com/apache/pulsar/pull/14024) -Fixed producerFuture.completeExceptionally not called before sendErrorResponse [#14025](https://github.com/apache/pulsar/pull/14025) -Make triggerOffload method async [#14027](https://github.com/apache/pulsar/pull/14027) -Make offloadStatus method async [#14029](https://github.com/apache/pulsar/pull/14029) -Make validateNamespacePolicyOperation method async in PulsarWebResource [#14033](https://github.com/apache/pulsar/pull/14033) -Make internalSkipMessages method async [#14045](https://github.com/apache/pulsar/pull/14045) -Make PersistentTopicsBase#internalSetBacklogQuota async [#14051](https://github.com/apache/pulsar/pull/14051) -Adjusted the validation for policy schemaCompatibilityStrategy [#14061](https://github.com/apache/pulsar/pull/14061) -Remove Persistent Topics v3 API - use custom media type instead [#14117](https://github.com/apache/pulsar/pull/14117) -Fix race condition in PulsarLedgerIdGenerator#generateShortLedgerId [#14118](https://github.com/apache/pulsar/pull/14118) -Optimized topic policy with HierarchyTopicPolicies about subscriptionDispatchRate [#14151](https://github.com/apache/pulsar/pull/14151) -Make `BrokerBase#deleteDynamicConfiguration` to pure async method [#14163](https://github.com/apache/pulsar/pull/14163) -Set default value of applied to false on topic policy [#14181](https://github.com/apache/pulsar/pull/14181) -Change broker producer fence log level [#14196](https://github.com/apache/pulsar/pull/14196) -Removed duplicated filter for UniformLoadShedder#findBundlesForUnloading [#14198](https://github.com/apache/pulsar/pull/14198) -Adjusted topic exists check logic in http lookup process [#14199](https://github.com/apache/pulsar/pull/14199) -Fixed NPE of internalExpireMessagesByTimestamp [#14243](https://github.com/apache/pulsar/pull/14243) -Fixed rackaware placement policy does not take effect after delete rack configuration [#14248](https://github.com/apache/pulsar/pull/14248) -Fixed print error log when server return redirect (http code 307) [#14259](https://github.com/apache/pulsar/pull/14259) -Optimized topic policy with HierarchyTopicPolicies about publishRate [#14267](https://github.com/apache/pulsar/pull/14267) -Fixed ack-hole and backlog for persistent-replicator [#14282](https://github.com/apache/pulsar/pull/14282) -Fixed NPE in internalSkipMessages [#14297](https://github.com/apache/pulsar/pull/14297) -Validate blank advertised listener name [#14306](https://github.com/apache/pulsar/pull/14306) -Let entries expire in the metadata caches [#14154](https://github.com/apache/pulsar/pull/14154) -Fixed transaction system topic loop creation [#12749](https://github.com/apache/pulsar/pull/12749) -Fixed topic transaction buffer handle null snapshot [#12758](https://github.com/apache/pulsar/pull/12758) -Optimized changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) +- [PIP 45] Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) +- [PIP 45] Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) +- [PIP 45] Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) +- [PIP 45] Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) +- [PIP 45] Removed old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) +- [PIP 45] Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) +- [PIP 45] Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) +- [PIP 45] Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) +- [PIP 45] Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) +- [PIP 45] Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) +- [PIP 45] Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) +- [PIP 45] Allow configuring configurationMetadataStore when initialize cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) +- [PIP 45] Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) +- [PIP 45] Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) +- [PIP 45] Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) +- [PIP 45] Fixed metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) +- Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) +- Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) +- Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) +- Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) +- Support split the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) +- Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) +- Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) +- Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) +- Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) +- Add uniform load shedder strategy to distribute traffic uniformly across brokers [#12902](https://github.com/apache/pulsar/pull/12902) +- Provide option to split bundle based on load [#12378](https://github.com/apache/pulsar/pull/12378) +- Use AuthorizationService#grantPermissionAsync to grant topic permission [#12515](https://github.com/apache/pulsar/pull/12515) +- Only store authentication data after authentication is complete [#12077](https://github.com/apache/pulsar/pull/12077) +- Allow to have different instances LocalMemoryMetadataStore that share the same state [#12390](https://github.com/apache/pulsar/pull/12390) +- Allow `GetTopicsOfNamespace` op with `consume` permission [#12600](https://github.com/apache/pulsar/pull/12600) +- Add local filesystem backend for package manager [#12708](https://github.com/apache/pulsar/pull/12708) +- Add stop replicator producer logic when start replicator cluster failed [#12724](https://github.com/apache/pulsar/pull/12724) +- Apply PolicyHierarchyValue to inactiveTopicPolicies [#12687](https://github.com/apache/pulsar/pull/12687) +- Fix can not get leader broker in follower brokers [#11353](https://github.com/apache/pulsar/pull/11353) +- Fix broker gc log options [#11285](https://github.com/apache/pulsar/pull/11285) +- Fix NPE of ZkBookieRackAffinityMapping [#11947](https://github.com/apache/pulsar/pull/11947) +- Fix prefix setting in JWT authn and avoid multi calls for the getProperty [#12132](https://github.com/apache/pulsar/pull/12132) +- Fix missed check exit code of stop before calling start [#12368](https://github.com/apache/pulsar/pull/12368) +- Fix getting children of parent nodes in LocalMemoryMetadataStore [#12491](https://github.com/apache/pulsar/pull/12491) +- Fix collection get bug in ResourceGroupService [#12499](https://github.com/apache/pulsar/pull/12499) +- Fix deadlock in metadata-store callback thread [#12753](https://github.com/apache/pulsar/pull/12753) +- Improve exceptions thrown when handling the schema resource [#12155](https://github.com/apache/pulsar/pull/12155) +- Trim topic name [#12453](https://github.com/apache/pulsar/pull/12453) +- Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic [#12658](https://github.com/apache/pulsar/pull/12658) +- Optimize isValidPath check in MetadataStore [#12663](https://github.com/apache/pulsar/pull/12663) +- Close connection after receiving unexpected SendCommand [#12780](https://github.com/apache/pulsar/pull/12780) +- Ensure cache is refreshed (and not just invalidated) after a store write [#12788](https://github.com/apache/pulsar/pull/12788) +- Optimize topicMaxMessageSize with topic local cache. [#12830](https://github.com/apache/pulsar/pull/12830) +- Optimize blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer [#12874](https://github.com/apache/pulsar/pull/12874) +- Only refresh metadata if path is already in cache after write. [#12896](https://github.com/apache/pulsar/pull/12896) +- Optimize put and get methods in AbstractMetadataStore [#12916](https://github.com/apache/pulsar/pull/12916) +- Fix zk-node leak of admin path [#12972](https://github.com/apache/pulsar/pull/12972) +- Optimize TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies [#13000](https://github.com/apache/pulsar/pull/13000) +- Fix race condition in FaultInjectionMetadataStore#programmedFailure [#13007](https://github.com/apache/pulsar/pull/13007) +- Fix usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts [#13025](https://github.com/apache/pulsar/pull/13025) +- Consider topics in pulsar/system namespace as system topics [#13050](https://github.com/apache/pulsar/pull/13050) +- Fix wrong result for looking up a non-exist topic by rest api [#13055](https://github.com/apache/pulsar/pull/13055) +- Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes [#13063](https://github.com/apache/pulsar/pull/13063) +- Make load-balancer config dynamic for the runtime tuning [#13074](https://github.com/apache/pulsar/pull/13074) +- Optimize TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https://github.com/apache/pulsar/pull/13082) +- Load balancer supports disabling max-session for bundle split [#13108](https://github.com/apache/pulsar/pull/13108) +- Optimize TopicPolicies#subscriptionTypesEnabled with HierarchyTopicPolicies [#13121](https://github.com/apache/pulsar/pull/13121) +- Log thread dump when Zookeeper session expires to help detect possible deadlocks [#13124](https://github.com/apache/pulsar/pull/13124) +- Add removeMaxConsumersPerSubscription method for v1 namespace [#13192](https://github.com/apache/pulsar/pull/13192) +- Fixe error value for 1MB in metrics [#13213](https://github.com/apache/pulsar/pull/13213) +- Optimize TopicPolicies#messageTTLInSeconds with HierarchyTopicPolicies [#13241](https://github.com/apache/pulsar/pull/13241) +- Add schema compatibility strategy on topic level [#13297](https://github.com/apache/pulsar/pull/13297) +- Optimize TopicPolicies#maxConsumerPerTopic with HierarchyTopicPolicies [#13361](https://github.com/apache/pulsar/pull/13361) +- Optimize TopicPolicies#replicationClusters with HierarchyTopicPolicies [#13495](https://github.com/apache/pulsar/pull/13495) +- Fix compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) +- Fix race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) +- Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) +- Return null instead of RestException when get bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) +- Fix failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) +- Optimize TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) +- Fix raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) +- Optimize TopicPolicies#maxUnackedMessagesOnConsumer with HierarchyTopicPolicies [#13618](https://github.com/apache/pulsar/pull/13618) +- Optimize TopicPolicies#delayedDelivery Enabled and TickTimeMillis with HierarchyTopicPolicies [#13649](https://github.com/apache/pulsar/pull/13649) +- Optimize TopicPolicies#compactionThreshold with HierarchyTopicPolicies [#13710](https://github.com/apache/pulsar/pull/13710) +- Add deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) +- Fix NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) +- Optimize deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) +- Fix call sync method in async rest api for `internalDeletePartitionedTopic` [#13805](https://github.com/apache/pulsar/pull/13805) +- Use shared executors for broker and geo-replication clients [#13839](https://github.com/apache/pulsar/pull/13839) +- Fix call sync method in async rest api for internalCreateSubscription. [#13873](https://github.com/apache/pulsar/pull/13873) +- Fix call sync method in async rest api for internalGetBacklogSizeByMessageId [#13871](https://github.com/apache/pulsar/pull/13871) +- Fix call sync method in async rest api for internalTriggerCompaction [#13853](https://github.com/apache/pulsar/pull/13853) +- Fix call sync method in async rest api for internalGetManagedLedgerInfo [#13847](https://github.com/apache/pulsar/pull/13847) +- Fix call sync method in async rest api for internalGetSubscriptions [#13846](https://github.com/apache/pulsar/pull/13846) +- Fix call sync method in async rest api for internalUnloadTopic [#13845](https://github.com/apache/pulsar/pull/13845) +- Fix call sync method in async rest api for internalGetLastMessageId [#13882](https://github.com/apache/pulsar/pull/13882) +- Fix call sync method in async rest api for internalExpireMessagesByTimestamp [#13880](https://github.com/apache/pulsar/pull/13880) +- Fix call sync method in async rest api for internalResetCursorOnPosition [#13879](https://github.com/apache/pulsar/pull/13879) +- Fix call sync method in async rest api for internalExpireMessagesByPosition [#13878](https://github.com/apache/pulsar/pull/13878) +- Fix call sync method in async rest api for internalGetMessageById. [#13876](https://github.com/apache/pulsar/pull/13876) +- Fix call sync method in async rest api for internalSetReplicatedSubscriptionStatus [#13887](https://github.com/apache/pulsar/pull/13887) +- Fix call sync method in async rest api for internalGetPartitionedStats [#13886](https://github.com/apache/pulsar/pull/13886) +- Fix call sync method in onPoliciesUpdate method [#13885](https://github.com/apache/pulsar/pull/13885) +- Fix call sync method in async rest api for internalGetPartitionedStatsInternal [#13884](https://github.com/apache/pulsar/pull/13884) +- Fix call sync method in async rest api for internalTerminatePartitionedTopic [#13890](https://github.com/apache/pulsar/pull/13890) +- Make PulsarAuthorizationProvider#grantPermissionAsync actually async [#13897](https://github.com/apache/pulsar/pull/13897) +- Fix no value present when updating dispatch rate [#13900](https://github.com/apache/pulsar/pull/13900) +- Fix call sync method in async rest api for internalSkipAllMessages [#13901](https://github.com/apache/pulsar/pull/13901) +- Optimize retentionPolicies with HierarchyTopicPolicies [#13909](https://github.com/apache/pulsar/pull/13909) +- Improve 403 response message wih consume permission on namespace [#13912](https://github.com/apache/pulsar/pull/13912) +- Fix arithmetic exception for uniform load shedder [#13914](https://github.com/apache/pulsar/pull/13914) +- Add threshold for each unload round for uniform load shedder [#13915](https://github.com/apache/pulsar/pull/13915) +- Fix call sync method in async rest api for internalDeleteSubscription #13884 [#13917](https://github.com/apache/pulsar/pull/13917) +- Change ``BrokersBase`` api ``getActiveBrokers`` and ``getLeaderBroker`` to pure async. [#13935](https://github.com/apache/pulsar/pull/13935) +- Do not create missing topic when loading namespace [#13948](https://github.com/apache/pulsar/pull/13948) +- Only reply to client when code completes producerFuture [#13949](https://github.com/apache/pulsar/pull/13949) +- Improve error logging for topic not found [#13950](https://github.com/apache/pulsar/pull/13950) +- Fix call sync method in async rest api for preValidation. [#13962](https://github.com/apache/pulsar/pull/13962) +- Fix call sync method in async rest api for internalSetReplicationClusters and internalRemoveReplicationClusters [#13961](https://github.com/apache/pulsar/pull/13961) +- Make validateTenantOperation method async in PulsarWebResource [#14008](https://github.com/apache/pulsar/pull/14008) +- Optimize maxUnackedMessagesOnSubscription with HierarchyTopicPolicies [#14011](https://github.com/apache/pulsar/pull/14011) +- Set default value of applied to false in getSchemaCompatibilityStrategy [#14012](https://github.com/apache/pulsar/pull/14012) +- Make validateTopicPolicyOperation method async in PulsarWebResource [#14024](https://github.com/apache/pulsar/pull/14024) +- Fix producerFuture.completeExceptionally not called before sendErrorResponse [#14025](https://github.com/apache/pulsar/pull/14025) +- Make triggerOffload method async [#14027](https://github.com/apache/pulsar/pull/14027) +- Make offloadStatus method async [#14029](https://github.com/apache/pulsar/pull/14029) +- Make validateNamespacePolicyOperation method async in PulsarWebResource [#14033](https://github.com/apache/pulsar/pull/14033) +- Make internalSkipMessages method async [#14045](https://github.com/apache/pulsar/pull/14045) +- Make PersistentTopicsBase#internalSetBacklogQuota async [#14051](https://github.com/apache/pulsar/pull/14051) +- Adjust the validation for policy schemaCompatibilityStrategy [#14061](https://github.com/apache/pulsar/pull/14061) +- Remove Persistent Topics v3 API - use custom media type instead [#14117](https://github.com/apache/pulsar/pull/14117) +- Fix race condition in PulsarLedgerIdGenerator#generateShortLedgerId [#14118](https://github.com/apache/pulsar/pull/14118) +- Optimize topic policy with HierarchyTopicPolicies about subscriptionDispatchRate [#14151](https://github.com/apache/pulsar/pull/14151) +- Make `BrokerBase#deleteDynamicConfiguration` to pure async method [#14163](https://github.com/apache/pulsar/pull/14163) +- Set default value of applied to false on topic policy [#14181](https://github.com/apache/pulsar/pull/14181) +- Change broker producer fence log level [#14196](https://github.com/apache/pulsar/pull/14196) +- Remove duplicated filter for UniformLoadShedder#findBundlesForUnloading [#14198](https://github.com/apache/pulsar/pull/14198) +- Adjust topic exists check logic in http lookup process [#14199](https://github.com/apache/pulsar/pull/14199) +- Fix NPE of internalExpireMessagesByTimestamp [#14243](https://github.com/apache/pulsar/pull/14243) +- Fix rackaware placement policy does not take effect after delete rack configuration [#14248](https://github.com/apache/pulsar/pull/14248) +- Fix print error log when server return redirect (http code 307) [#14259](https://github.com/apache/pulsar/pull/14259) +- Optimize topic policy with HierarchyTopicPolicies about publishRate [#14267](https://github.com/apache/pulsar/pull/14267) +- Fix ack-hole and backlog for persistent-replicator [#14282](https://github.com/apache/pulsar/pull/14282) +- Fix NPE in internalSkipMessages [#14297](https://github.com/apache/pulsar/pull/14297) +- Validate blank advertised listener name [#14306](https://github.com/apache/pulsar/pull/14306) +- Let entries expire in the metadata caches [#14154](https://github.com/apache/pulsar/pull/14154) +- Fix transaction system topic loop creation [#12749](https://github.com/apache/pulsar/pull/12749) +- Fix topic transaction buffer handle null snapshot [#12758](https://github.com/apache/pulsar/pull/12758) +- Optimize changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) ### Clients -[Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) -[Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) -[Java] Fixed the producer OOM if got exception while add message to batch container [#12170](https://github.com/apache/pulsar/pull/12170) -[Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) -[Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) -[Java] Fixed confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) -[Java] Fixed parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) -[Java] Added getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) -[Java] Allowed config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) -[Java] Support add custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) -[Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) -[Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) -[Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) -[Java] Log producer batchSize and msgSize percentiles [#14229](https://github.com/apache/pulsar/pull/14229) -[C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) -[C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) -[C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) -[C++] Support more pulsar command name and version in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) -[C++] Optimize MessageBuilder and SharedBuffer to avoid unnecessary memory copy [#13293](https://github.com/apache/pulsar/pull/13293) -[C++] Fixed in Apple Silicon macOS the clang-format cannot find [#13333](https://github.com/apache/pulsar/pull/13333) -[C++] Added clang-format check for Pulsar Wireshark dissector [#13349](https://github.com/apache/pulsar/pull/13349) -[C++] Adjust clang-format search names [#13369](https://github.com/apache/pulsar/pull/13369) -[C++] Fixed Wireshark dissector decode send command metadata behavior [#13471](https://github.com/apache/pulsar/pull/13471) -[C++] PIP 37: Support large message size [#13627](https://github.com/apache/pulsar/pull/13627) -[C++] Fix the consumer configuration inconsistency with Java client [#14070](https://github.com/apache/pulsar/pull/14070) -[Python] Provide __str__ operator for BytesSchema [#12593](https://github.com/apache/pulsar/pull/12593) -[Python] Support is_connected in Python Client [#13662](https://github.com/apache/pulsar/pull/13662) -[C] Add missing includes in reader_configuration.h [#12966](https://github.com/apache/pulsar/pull/12966) -[C] Add pulsar_client_subscribe_multi_topics and pulsar_client_subscribe_pattern [#12965](https://github.com/apache/pulsar/pull/12965) +- [Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) +- [Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) +- [Java] Fix the producer OOM if got exception while add message to batch container [#12170](https://github.com/apache/pulsar/pull/12170) +- [Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) +- [Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) +- [Java] Fix confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) +- [Java] Fix parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) +- [Java] Add getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) +- [Java] Allow config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) +- [Java] Support add custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) +- [Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) +- [Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) +- [Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) +- [Java] Log producer batchSize and msgSize percentiles [#14229](https://github.com/apache/pulsar/pull/14229) +- [C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) +- [C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) +- [C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) +- [C++] Support more pulsar command name and version in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) +- [C++] Optimize MessageBuilder and SharedBuffer to avoid unnecessary memory copy [#13293](https://github.com/apache/pulsar/pull/13293) +- [C++] Fix in Apple Silicon macOS the clang-format cannot find [#13333](https://github.com/apache/pulsar/pull/13333) +- [C++] Add clang-format check for Pulsar Wireshark dissector [#13349](https://github.com/apache/pulsar/pull/13349) +- [C++] Adjust clang-format search names [#13369](https://github.com/apache/pulsar/pull/13369) +- [C++] Fix Wireshark dissector decode send command metadata behavior [#13471](https://github.com/apache/pulsar/pull/13471) +- [C++] PIP 37: Support large message size [#13627](https://github.com/apache/pulsar/pull/13627) +- [C++] Fix the consumer configuration inconsistency with Java client [#14070](https://github.com/apache/pulsar/pull/14070) +- [Python] Provide __str__ operator for BytesSchema [#12593](https://github.com/apache/pulsar/pull/12593) +- [Python] Support is_connected in Python Client [#13662](https://github.com/apache/pulsar/pull/13662) +- [C] Add missing includes in reader_configuration.h [#12966](https://github.com/apache/pulsar/pull/12966) +- [C] Add pulsar_client_subscribe_multi_topics and pulsar_client_subscribe_pattern [#12965](https://github.com/apache/pulsar/pull/12965) ### Pulsar IO and Pulsar Functions -[Functions] Prevent NPE while stopping a non started Pulsar LogAppender [#12643](https://github.com/apache/pulsar/pull/12643) -[Functions] Allow configuring different implementations for Pulsar functions state store [#12646](https://github.com/apache/pulsar/pull/12646) -[Functions] Clean os even when statusFuture complete exceptionally [#12767](https://github.com/apache/pulsar/pull/12767) -[Functions] Override inactive_topic_policies in Pulsar Functions namespace creation [#13048](https://github.com/apache/pulsar/pull/13048) -[Functions] Added possibility to pass additional JVM arguments to the function JVM (additionalJavaRuntimeArguments) [#13282](https://github.com/apache/pulsar/pull/13282) -[Functions] Fixed `getTlsTrustChainBytes` not work when functions worker not run with broker [#13875](https://github.com/apache/pulsar/pull/13875) -[Functions] Fixed distributed log metadata not correctly initialized [#13891](https://github.com/apache/pulsar/pull/13891) -[Functions] Removed sensitive information from log [#14159](https://github.com/apache/pulsar/pull/14159) -[IO Connector] Pass client builder if no service url provided to debezium connector [#12145](https://github.com/apache/pulsar/pull/12145) -[IO Connector] Fixed: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test [12374](https://github.com/apache/pulsar/pull/12374) -[IO Connector] Fixed the marshal and unmarshal the sink config [#12625](https://github.com/apache/pulsar/pull/12625) -[IO Connector] Support rename the file has been processed [#13373](https://github.com/apache/pulsar/pull/13373) -[IO Connector] Support Fixed and ENUM datatypes for ElasticSearch Sink [#13800](https://github.com/apache/pulsar/pull/13800) -[IO Connector] Implement --retain-key-ordering (KEY_SHARED subscription) for Sinks [#14083](https://github.com/apache/pulsar/pull/14083) +- [Functions] Prevent NPE while stopping a non started Pulsar LogAppender [#12643](https://github.com/apache/pulsar/pull/12643) +- [Functions] Allow configuring different implementations for Pulsar functions state store [#12646](https://github.com/apache/pulsar/pull/12646) +- [Functions] Clean os even when statusFuture complete exceptionally [#12767](https://github.com/apache/pulsar/pull/12767) +- [Functions] Override inactive_topic_policies in Pulsar Functions namespace creation [#13048](https://github.com/apache/pulsar/pull/13048) +- [Functions] Add possibility to pass additional JVM arguments to the function JVM (additionalJavaRuntimeArguments) [#13282](https://github.com/apache/pulsar/pull/13282) +- [Functions] Fix `getTlsTrustChainBytes` not work when functions worker not run with broker [#13875](https://github.com/apache/pulsar/pull/13875) +- [Functions] Fix distributed log metadata not correctly initialized [#13891](https://github.com/apache/pulsar/pull/13891) +- [Functions] Remove sensitive information from log [#14159](https://github.com/apache/pulsar/pull/14159) +- [IO Connector] Pass client builder if no service url provided to debezium connector [#12145](https://github.com/apache/pulsar/pull/12145) +- [IO Connector] Fix: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test [12374](https://github.com/apache/pulsar/pull/12374) +- [IO Connector] Fix the marshal and unmarshal the sink config [#12625](https://github.com/apache/pulsar/pull/12625) +- [IO Connector] Support rename the file has been processed [#13373](https://github.com/apache/pulsar/pull/13373) +- [IO Connector] Support Fixed and ENUM datatypes for ElasticSearch Sink [#13800](https://github.com/apache/pulsar/pull/13800) +- [IO Connector] Implement --retain-key-ordering (KEY_SHARED subscription) for Sinks [#14083](https://github.com/apache/pulsar/pull/14083) ### Observability -[Broker] Exposed broker bundles metrics to prometheus [#12366](https://github.com/apache/pulsar/pull/12366) -[Broker] Added publishRateLimitedTimes to topic metrics [#13538](https://github.com/apache/pulsar/pull/13538) -[Broker] Fixed bundle metrics would overwrite loadbalance metrics [#13641](https://github.com/apache/pulsar/pull/13641) -[Broker] Fixed managed cursor acknowledgment state metric names [#13844](https://github.com/apache/pulsar/pull/13844) -[Java Client] Added pending-queue size metrics to producer stats [#12674](https://github.com/apache/pulsar/pull/12674) +- [Broker] Expose broker bundles metrics to prometheus [#12366](https://github.com/apache/pulsar/pull/12366) +- [Broker] Add publishRateLimitedTimes to topic metrics [#13538](https://github.com/apache/pulsar/pull/13538) +- [Broker] Fix bundle metrics would overwrite loadbalance metrics [#13641](https://github.com/apache/pulsar/pull/13641) +- [Broker] Fix managed cursor acknowledgment state metric names [#13844](https://github.com/apache/pulsar/pull/13844) +- [Java Client] Add pending-queue size metrics to producer stats [#12674](https://github.com/apache/pulsar/pull/12674) ### CLI -[Pulsar Admin] Support get a mapping of brokers to partitioned topics [#11763](https://github.com/apache/pulsar/pull/11763) -[Pulsar Admin] Support terminate a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) -[Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) -[Pulsar Admin] Added remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) -[Pulsar Admin] Fixed output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) -[Pulsar Admin] Added a metric to get the earliest time in the backlog [#12523](https://github.com/apache/pulsar/pull/12523) -[Pulsar Admin] Reduced severity of log "refreshing key manager" in KeyManagerProxy [#12594](https://github.com/apache/pulsar/pull/12594) -[Pulsar Admin] Support to get list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) -[Pulsar Admin] Added get-replicated-subscription-status command for topic [#12891](https://github.com/apache/pulsar/pull/12891) -[Pulsar Admin] Added remove-subscription-types-enabled command for topic [#12983](https://github.com/apache/pulsar/pull/12983) -[Pulsar Admin] Validate the size options in cmd for topic and namespace [#13002](https://github.com/apache/pulsar/pull/13002) -[Pulsar Admin] Support subscription across multiple clusters [#13482](https://github.com/apache/pulsar/pull/13482) -[Pulsar Admin] Support deduplication across multiple clusters [#13487](https://github.com/apache/pulsar/pull/13487) -[Pulsar Admin] Support message TTL across multiple clusters [#13484](https://github.com/apache/pulsar/pull/13484) -[Pulsar Admin] Support persistence policies across multiple clusters [#13483](https://github.com/apache/pulsar/pull/13483) -[Pulsar Admin] Support publish rate across multiple clusters [#13496](https://github.com/apache/pulsar/pull/13496) -[Pulsar Admin] Support max consumers across multiple clusters [#13521](https://github.com/apache/pulsar/pull/13521) -[Pulsar Admin] Support max producer across multiple clusters [#13519](https://github.com/apache/pulsar/pull/13519) -[Pulsar Admin] Support compaction threshold across multiple clusters [#13513](https://github.com/apache/pulsar/pull/13513) -[Pulsar Admin] Support offload policies across multiple clusters [#13534](https://github.com/apache/pulsar/pull/13534) -[Pulsar Admin] Support max unacked messages per consumer across multiple clusters [#13547](https://github.com/apache/pulsar/pull/13547) -[Pulsar Admin] Support max message size support across multiple clusters [#13579](https://github.com/apache/pulsar/pull/13579) -[Pulsar Admin] Support deduplication snapshot interval across multiple clusters [#13578](https://github.com/apache/pulsar/pull/13578) -[Pulsar Admin] Support delayed delivery policy across multiple clusters [#13550](https://github.com/apache/pulsar/pull/13550) -[Pulsar Admin] Support max unacked messages on subscription across multiple clusters [#13549](https://github.com/apache/pulsar/pull/13549) -[Pulsar Admin] Support replicator dispatch rate across multiple clusters [#13624](https://github.com/apache/pulsar/pull/13624) -[Pulsar Admin] Support max subscriptions per topic across multiple clusters [#13623](https://github.com/apache/pulsar/pull/13623) -[Pulsar Admin] Move schema compatibility strategy cmd from topics to topicPolicies [#14225](https://github.com/apache/pulsar/pull/14225) -[Pulsar CLI] Added restart command to pulsar-daemon [#12279](https://github.com/apache/pulsar/pull/12279) -[Pulsar CLI] Print log for standalone when configuration is failed to load [#12280](https://github.com/apache/pulsar/pull/12280) -[Client Tool] Support Disabling Replication [#13659](https://github.com/apache/pulsar/pull/13659) -[Perf Tool] Added auth for transaction perf [#14271](https://github.com/apache/pulsar/pull/14271) +- [Pulsar Admin] Support get a mapping of brokers to partitioned topics [#11763](https://github.com/apache/pulsar/pull/11763) +- [Pulsar Admin] Support terminate a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) +- [Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) +- [Pulsar Admin] Add remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) +- [Pulsar Admin] Fix output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) +- [Pulsar Admin] Add a metric to get the earliest time in the backlog [#12523](https://github.com/apache/pulsar/pull/12523) +- [Pulsar Admin] Reduce severity of log "refreshing key manager" in KeyManagerProxy [#12594](https://github.com/apache/pulsar/pull/12594) +- [Pulsar Admin] Support to get list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) +- [Pulsar Admin] Add get-replicated-subscription-status command for topic [#12891](https://github.com/apache/pulsar/pull/12891) +- [Pulsar Admin] Add remove-subscription-types-enabled command for topic [#12983](https://github.com/apache/pulsar/pull/12983) +- [Pulsar Admin] Validate the size options in cmd for topic and namespace [#13002](https://github.com/apache/pulsar/pull/13002) +- [Pulsar Admin] Support subscription across multiple clusters [#13482](https://github.com/apache/pulsar/pull/13482) +- [Pulsar Admin] Support deduplication across multiple clusters [#13487](https://github.com/apache/pulsar/pull/13487) +- [Pulsar Admin] Support message TTL across multiple clusters [#13484](https://github.com/apache/pulsar/pull/13484) +- [Pulsar Admin] Support persistence policies across multiple clusters [#13483](https://github.com/apache/pulsar/pull/13483) +- [Pulsar Admin] Support publish rate across multiple clusters [#13496](https://github.com/apache/pulsar/pull/13496) +- [Pulsar Admin] Support max consumers across multiple clusters [#13521](https://github.com/apache/pulsar/pull/13521) +- [Pulsar Admin] Support max producer across multiple clusters [#13519](https://github.com/apache/pulsar/pull/13519) +- [Pulsar Admin] Support compaction threshold across multiple clusters [#13513](https://github.com/apache/pulsar/pull/13513) +- [Pulsar Admin] Support offload policies across multiple clusters [#13534](https://github.com/apache/pulsar/pull/13534) +- [Pulsar Admin] Support max unacked messages per consumer across multiple clusters [#13547](https://github.com/apache/pulsar/pull/13547) +- [Pulsar Admin] Support max message size support across multiple clusters [#13579](https://github.com/apache/pulsar/pull/13579) +- [Pulsar Admin] Support deduplication snapshot interval across multiple clusters [#13578](https://github.com/apache/pulsar/pull/13578) +- [Pulsar Admin] Support delayed delivery policy across multiple clusters [#13550](https://github.com/apache/pulsar/pull/13550) +- [Pulsar Admin] Support max unacked messages on subscription across multiple clusters [#13549](https://github.com/apache/pulsar/pull/13549) +- [Pulsar Admin] Support replicator dispatch rate across multiple clusters [#13624](https://github.com/apache/pulsar/pull/13624) +- [Pulsar Admin] Support max subscriptions per topic across multiple clusters [#13623](https://github.com/apache/pulsar/pull/13623) +- [Pulsar Admin] Move schema compatibility strategy cmd from topics to topicPolicies [#14225](https://github.com/apache/pulsar/pull/14225) +- [Pulsar CLI] Add restart command to pulsar-daemon [#12279](https://github.com/apache/pulsar/pull/12279) +- [Pulsar CLI] Print log for standalone when configuration is failed to load [#12280](https://github.com/apache/pulsar/pull/12280) +- [Client Tool] Support Disabling Replication [#13659](https://github.com/apache/pulsar/pull/13659) +- [Perf Tool] Add auth for transaction perf [#14271](https://github.com/apache/pulsar/pull/14271) ### Others -[Proxy] Fixed Pulsar Proxy to re-use authentication instance [#12245](https://github.com/apache/pulsar/pull/12245) -[Proxy] Fixed auto-cert refresh when proxy connects to broker [#14130](https://github.com/apache/pulsar/pull/14130) -[WebSocket] Fixed the batch message ack [#12530](https://github.com/apache/pulsar/pull/12530) -[Package Management] check service status before run commands [#12847](https://github.com/apache/pulsar/pull/12847) -[Package Management] Add package management filesystem storage into the distribution lib [#13202](https://github.com/apache/pulsar/pull/13202) -[Package Management] Fix the filesystem storage failure [#13218](https://github.com/apache/pulsar/pull/13218) -[Schema] Fix pulsar use json or avro primitive schema [#12886](https://github.com/apache/pulsar/pull/12886) -[Tiered Storage] Fixed FileSystemManagedLedgerOffloader can not cleanup outdated ledger [#12309](https://github.com/apache/pulsar/pull/12309) -[Tiered Storage] Fixed NoClassDefFoundError: com/google/inject/AbstractModule in pulsar-io/batch-data-generator and Jcloud offloader [#14150](https://github.com/apache/pulsar/pull/14150) -Added lua wireshark for pulsar [#13564](https://github.com/apache/pulsar/pull/13564) -Fixed missing new line at the end of proxy.conf [#14359](https://github.com/apache/pulsar/pull/14359) +- [Proxy] Fix Pulsar Proxy to re-use authentication instance [#12245](https://github.com/apache/pulsar/pull/12245) +- [Proxy] Fix auto-cert refresh when proxy connects to broker [#14130](https://github.com/apache/pulsar/pull/14130) +- [WebSocket] Fix the batch message ack [#12530](https://github.com/apache/pulsar/pull/12530) +- [Package Management] check service status before run commands [#12847](https://github.com/apache/pulsar/pull/12847) +- [Package Management] Add package management filesystem storage into the distribution lib [#13202](https://github.com/apache/pulsar/pull/13202) +- [Package Management] Fix the filesystem storage failure [#13218](https://github.com/apache/pulsar/pull/13218) +- [Schema] Fix pulsar use json or avro primitive schema [#12886](https://github.com/apache/pulsar/pull/12886) +- [Tiered Storage] Fix FileSystemManagedLedgerOffloader can not cleanup outdated ledger [#12309](https://github.com/apache/pulsar/pull/12309) +- [Tiered Storage] Fix NoClassDefFoundError: com/google/inject/AbstractModule in pulsar-io/batch-data-generator and Jcloud offloader [#14150](https://github.com/apache/pulsar/pull/14150) +- Add lua wireshark for pulsar [#13564](https://github.com/apache/pulsar/pull/13564) +- Fix missing new line at the end of proxy.conf [#14359](https://github.com/apache/pulsar/pull/14359) ### Library updates -Upgraded function's Go client version to v0.7.0 [#12839](https://github.com/apache/pulsar/pull/12839) -Upgraded dependencies (guava and what brought in older guava) to get rid of the guava-related CVE-2018-10237 and CVE-2020-8908 [#13716](https://github.com/apache/pulsar/pull/13716) -Use dependencyManagement to enforce snakeyaml version to 1.30 [#13722](https://github.com/apache/pulsar/pull/13722) -Upgraded dependencies to get rid of pulsar-io/jdbc related CVE-2020-13692 [#13753](https://github.com/apache/pulsar/pull/13753) -Removed --illegal-access errors resulting from Google Guice (upgrade to 5.0.1 and JClouds to 2.4.0) [#13810](https://github.com/apache/pulsar/pull/13810) -Upgraded jakarta.el to 3.0.4 to get rid of CVE-2021-28170 [#13943](https://github.com/apache/pulsar/pull/13943) -Upgraded Netty to 4.1.73.Final [#13981](https://github.com/apache/pulsar/pull/13981) -Removed net.jodah.failsafe dependency (fix JDK17 build) [#14124](https://github.com/apache/pulsar/pull/14124) -Upgraded netty version to 4.1.74.Final [#14257](https://github.com/apache/pulsar/pull/14257) +- Upgrade function's Go client version to v0.7.0 [#12839](https://github.com/apache/pulsar/pull/12839) +- Upgrade dependencies (guava and what brought in older guava) to get rid of the guava-related CVE-2018-10237 and CVE-2020-8908 [#13716](https://github.com/apache/pulsar/pull/13716) +- Use dependencyManagement to enforce snakeyaml version to 1.30 [#13722](https://github.com/apache/pulsar/pull/13722) +- Upgrade dependencies to get rid of pulsar-io/jdbc related CVE-2020-13692 [#13753](https://github.com/apache/pulsar/pull/13753) +- Remove --illegal-access errors resulting from Google Guice (upgrade to 5.0.1 and JClouds to 2.4.0) [#13810](https://github.com/apache/pulsar/pull/13810) +- Upgrade jakarta.el to 3.0.4 to get rid of CVE-2021-28170 [#13943](https://github.com/apache/pulsar/pull/13943) +- Upgrade Netty to 4.1.73.Final [#13981](https://github.com/apache/pulsar/pull/13981) +- Remove net.jodah.failsafe dependency (fix JDK17 build) [#14124](https://github.com/apache/pulsar/pull/14124) +- Upgrade netty version to 4.1.74.Final [#14257](https://github.com/apache/pulsar/pull/14257) ### 2.9.1 #### 2021-12-20 From 88ef3d5412dee7e23a46492c12a0094574314dd1 Mon Sep 17 00:00:00 2001 From: penghui Date: Mon, 11 Apr 2022 23:34:57 +0800 Subject: [PATCH 05/20] Fix tests. --- site2/website/release-notes.md | 1 - 1 file changed, 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 27c50b9f3e6b6..dd8d6af6c118b 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -496,7 +496,6 @@ - Upgrade Gson version 2.8.6 to 2.8.9 [13610](https://github.com/apache/pulsar/pull/13610) - Upgrade commons-cli to 1.5.0 [14094](https://github.com/apache/pulsar/pull/14094) - Bump netty version to 4.1.74.Final [14257](https://github.com/apache/pulsar/pull/14257) ->>>>>>> apache/master ### 2.9.1 #### 2021-12-20 From aa78aa36d82e168376de7ca6a263d6965303e9c8 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:58:48 +0800 Subject: [PATCH 06/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index dd8d6af6c118b..4e36ae3e83897 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -45,7 +45,7 @@ - Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) - Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) - Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) - - Allow configuring configurationMetadataStore when initialize cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) + - Allow configuring configurationMetadataStore when initializing cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) - Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) - Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) - Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) From 223c687e110d5e9cb8b305314a072d44c1e2c342 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:58:58 +0800 Subject: [PATCH 07/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 4e36ae3e83897..8cba488319b14 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -42,7 +42,7 @@ - Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) - Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) - Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) - - Allow to configure metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) + - Allow configuring metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) - Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) - Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) - Allow configuring configurationMetadataStore when initializing cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) From cfb364b8b162d9e8aafc0ccb733d9b180f431eff Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:07 +0800 Subject: [PATCH 08/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 8cba488319b14..517cf315a7d8a 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -43,7 +43,7 @@ - Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) - Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) - Allow configuring metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) - - Allow to configure metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) + - Allow configuring metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) - Add configuration metadata store url setting in WebSocket [#13786](https://github.com/apache/pulsar/pull/13786) - Allow configuring configurationMetadataStore when initializing cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) - Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) From 04983b4bcc197da7cd962a36f19dc533b6c7895e Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:15 +0800 Subject: [PATCH 09/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 517cf315a7d8a..6e0eaa23d987e 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -107,7 +107,7 @@ - Fix compatibility issue with other metadata store implementations in resources [#13394](https://github.com/apache/pulsar/pull/13394) - Fix race condition in stopping replicator while it is starting [#13412](https://github.com/apache/pulsar/pull/13412) - Support backlog quota across multiple clusters [#13445](https://github.com/apache/pulsar/pull/13445) -- Return null instead of RestException when get bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) +- Return null instead of RestException when getting bookieAffinityGroup [#13462](https://github.com/apache/pulsar/pull/13462) - Fix failed to get Partitioned metadata for health checking [#13525](https://github.com/apache/pulsar/pull/13525) - Optimize TopicPolicies#maxConsumersPerSubscription with HierarchyTopicPolicies [#13548](https://github.com/apache/pulsar/pull/13548) - Fix raw use of generic types in OwnershipCache [#13592](https://github.com/apache/pulsar/pull/13592) From 8302d67eb1725feca6267718ec61a8f20c031aa2 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:25 +0800 Subject: [PATCH 10/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 6e0eaa23d987e..03cfa25fe5552 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -184,7 +184,7 @@ ### Clients - [Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) - [Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) -- [Java] Fix the producer OOM if got exception while add message to batch container [#12170](https://github.com/apache/pulsar/pull/12170) +- [Java] Fix the producer OOM if got an exception while adding messages to batch container [#12170](https://github.com/apache/pulsar/pull/12170) - [Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) - [Java] Support negative ack redelivery backoff [#12566](https://github.com/apache/pulsar/pull/12566) - [Java] Fix confusing logs in UnAckedMessageTracker [#13017](https://github.com/apache/pulsar/pull/13017) From 671cf6b339f5aaba1c2b0832d6118c80c6f55dff Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:37 +0800 Subject: [PATCH 11/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 03cfa25fe5552..4344609016aab 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -199,7 +199,7 @@ - [C++] Add Wireshark cmake and fix build with latest Wireshark [#13236](https://github.com/apache/pulsar/pull/13236) - [C++] Wireshark Pulsar dissector naming replace yahoo with apache [#13251](https://github.com/apache/pulsar/pull/13251) - [C++] Support arm64 optimized CRC32c hardware-instructions [#13246](https://github.com/apache/pulsar/pull/13246) -- [C++] Support more pulsar command name and version in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) +- [C++] Support more pulsar command names and versions in Wireshark dissector [#13286](https://github.com/apache/pulsar/pull/13286) - [C++] Optimize MessageBuilder and SharedBuffer to avoid unnecessary memory copy [#13293](https://github.com/apache/pulsar/pull/13293) - [C++] Fix in Apple Silicon macOS the clang-format cannot find [#13333](https://github.com/apache/pulsar/pull/13333) - [C++] Add clang-format check for Pulsar Wireshark dissector [#13349](https://github.com/apache/pulsar/pull/13349) From d3b0721f83cf9010b2cd16826dc242b7bed0393c Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:45 +0800 Subject: [PATCH 12/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 4344609016aab..b4cfe4f28c14c 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -54,7 +54,7 @@ - Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) - Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) - Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) -- Support split the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) +- Support splitting the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) - Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) - Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) - Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) From e1210967c1c1698d4b8822cdcf1b8da9fcb8bcae Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 21:59:52 +0800 Subject: [PATCH 13/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index b4cfe4f28c14c..c825df2f162b3 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -55,7 +55,7 @@ - Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173) - Support configuration to rate-limit dispatching on batch message [#12294](https://github.com/apache/pulsar/pull/12294) - Support splitting the largest bundle of the namespace [#12361](https://github.com/apache/pulsar/pull/12361) -- Support create subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) +- Support creating a subscription by specifying the earliest or latest position [#12872](https://github.com/apache/pulsar/pull/12872) - Support roll-over ledgers for inactive topics [#13073](https://github.com/apache/pulsar/pull/13073) - Support graceful shutdown for Broker [#14114](https://github.com/apache/pulsar/pull/14114) - Transparent batching of ZK operations [#13043](https://github.com/apache/pulsar/pull/13043) From 27a4fd2b97dec4d7e1a6a9a6e19ef394e01b5abf Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:00 +0800 Subject: [PATCH 14/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index c825df2f162b3..54a97dfea4bf2 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -182,7 +182,7 @@ - Optimize changeToCloseState method [#14277](https://github.com/apache/pulsar/pull/14277) ### Clients -- [Java] Support create a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) +- [Java] Support creating a consumer in the paused state [#11974](https://github.com/apache/pulsar/pull/11974) - [Java] Support passing existing executor providers to the client [#12037](https://github.com/apache/pulsar/pull/12037) - [Java] Fix the producer OOM if got an exception while adding messages to batch container [#12170](https://github.com/apache/pulsar/pull/12170) - [Java] Allow to config client allocator out of memory policy [#12200](https://github.com/apache/pulsar/pull/12200) From 300c09d0cbb3419238bbee3724a9b44cc6296195 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:07 +0800 Subject: [PATCH 15/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 54a97dfea4bf2..9417a0bb4fab3 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -191,7 +191,7 @@ - [Java] Fix parseProtobufSchema method will be called two times [#13163](https://github.com/apache/pulsar/pull/13163) - [Java] Add getNumPartitions method into PartitionedProducerImpl [#13239](https://github.com/apache/pulsar/pull/13239) - [Java] Allow config client dns bind addr and port [#13390](https://github.com/apache/pulsar/pull/13390) -- [Java] Support add custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) +- [Java] Support adding custom properties for the reconsumeLater interface [#13461](https://github.com/apache/pulsar/pull/13461) - [Java] Allow Client Builder set Dnslookup params [#13503](https://github.com/apache/pulsar/pull/13503) - [Java] Avoid repeatedly set startMessageIdData to null for ConsumerImpl [#13606](https://github.com/apache/pulsar/pull/13606) - [Java] Let the 'properties' to be empty for ConsumerBuilder and ProducerBuilder [#14074](https://github.com/apache/pulsar/pull/14074) From 200ad7d26ee9aeab0c6ecd800efb3db5ccf7b3a7 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:15 +0800 Subject: [PATCH 16/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 9417a0bb4fab3..c26bfcf5c9abc 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -225,7 +225,7 @@ - [IO Connector] Pass client builder if no service url provided to debezium connector [#12145](https://github.com/apache/pulsar/pull/12145) - [IO Connector] Fix: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test [12374](https://github.com/apache/pulsar/pull/12374) - [IO Connector] Fix the marshal and unmarshal the sink config [#12625](https://github.com/apache/pulsar/pull/12625) -- [IO Connector] Support rename the file has been processed [#13373](https://github.com/apache/pulsar/pull/13373) +- [IO Connector] Support renaming the file that has been processed [#13373](https://github.com/apache/pulsar/pull/13373) - [IO Connector] Support Fixed and ENUM datatypes for ElasticSearch Sink [#13800](https://github.com/apache/pulsar/pull/13800) - [IO Connector] Implement --retain-key-ordering (KEY_SHARED subscription) for Sinks [#14083](https://github.com/apache/pulsar/pull/14083) From 832410ae79cc8a604a5e2793028b52d79f1b1a77 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:21 +0800 Subject: [PATCH 17/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index c26bfcf5c9abc..0471a1db9579f 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -239,7 +239,7 @@ ### CLI - [Pulsar Admin] Support get a mapping of brokers to partitioned topics [#11763](https://github.com/apache/pulsar/pull/11763) -- [Pulsar Admin] Support terminate a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) +- [Pulsar Admin] Support the termination of a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) - [Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) - [Pulsar Admin] Add remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) - [Pulsar Admin] Fix output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) From d6254929338f118e8d6a0fbcbdd4f6a6e4459f35 Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:27 +0800 Subject: [PATCH 18/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 0471a1db9579f..1010afac0f50f 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -238,7 +238,7 @@ - [Java Client] Add pending-queue size metrics to producer stats [#12674](https://github.com/apache/pulsar/pull/12674) ### CLI -- [Pulsar Admin] Support get a mapping of brokers to partitioned topics [#11763](https://github.com/apache/pulsar/pull/11763) +- [Pulsar Admin] Support a mapping from brokers to partitioned topics that each broker serves [#11763](https://github.com/apache/pulsar/pull/11763) - [Pulsar Admin] Support the termination of a partitioned topic [#11893](https://github.com/apache/pulsar/pull/11893) - [Pulsar Admin] Update command descriptions from old 'property/cluster/namespace' format to current 'tenant/namespace' format [#10485](https://github.com/apache/pulsar/pull/10485) - [Pulsar Admin] Add remove-subscription-types-enabled command for namespace [#12392](https://github.com/apache/pulsar/pull/12392) From e156f5c0498886386c2abdf930d0b2db5563213f Mon Sep 17 00:00:00 2001 From: lipenghui Date: Tue, 12 Apr 2022 22:00:36 +0800 Subject: [PATCH 19/20] Update site2/website/release-notes.md Co-authored-by: momo-jun <60642177+momo-jun@users.noreply.github.com> --- site2/website/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 1010afac0f50f..7e6acc4a8201f 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -245,7 +245,7 @@ - [Pulsar Admin] Fix output format of string by pulsar-admin command [#11878](https://github.com/apache/pulsar/pull/11878) - [Pulsar Admin] Add a metric to get the earliest time in the backlog [#12523](https://github.com/apache/pulsar/pull/12523) - [Pulsar Admin] Reduce severity of log "refreshing key manager" in KeyManagerProxy [#12594](https://github.com/apache/pulsar/pull/12594) -- [Pulsar Admin] Support to get list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) +- [Pulsar Admin] Support getting a list of topics under a namespace bundle [#12632](https://github.com/apache/pulsar/pull/12632) - [Pulsar Admin] Add get-replicated-subscription-status command for topic [#12891](https://github.com/apache/pulsar/pull/12891) - [Pulsar Admin] Add remove-subscription-types-enabled command for topic [#12983](https://github.com/apache/pulsar/pull/12983) - [Pulsar Admin] Validate the size options in cmd for topic and namespace [#13002](https://github.com/apache/pulsar/pull/13002) From 0e7920275400fefbfe5b52d8f88be16725f342ce Mon Sep 17 00:00:00 2001 From: penghui Date: Thu, 14 Apr 2022 09:58:51 +0800 Subject: [PATCH 20/20] Apply comments. --- site2/website/release-notes.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/site2/website/release-notes.md b/site2/website/release-notes.md index 7e6acc4a8201f..10b8de7c3ab5d 100644 --- a/site2/website/release-notes.md +++ b/site2/website/release-notes.md @@ -10,14 +10,14 @@ - Reduce the time it takes for namespace bundle unloading to time out [#12995](https://github.com/apache/pulsar/pull/12995) - Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) [#13272](https://github.com/apache/pulsar/pull/13272) - [PIP 118] Do not restart brokers when ZooKeeper session expires as default [#13341](https://github.com/apache/pulsar/pull/13341) -- [PIP 119] Enabled consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) -- [PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) +- [PIP 119] Enable consistent hashing by default on KeyShared subscriptions dispatcher [#13352](https://github.com/apache/pulsar/pull/13352) +- [PIP 120] Enable client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) - Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) - [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) - Fix netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) ### PIPs -- [PIP 79] Added lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) +- [PIP 79] Add lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) - [PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) - [PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) - [PIP 92] Topic policy across multiple clusters [#12517](https://github.com/apache/pulsar/pull/12517) @@ -34,13 +34,13 @@ ### Broker - [PIP 45] Pluggable metadata interface - - Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) + - Add BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) - Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) - - Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) + - Convert BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) - Allow to configure metadata store URL in broker.conf [#13077](https://github.com/apache/pulsar/pull/13077) - - Removed old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) + - Remove old ZK caches implementations [#13075](https://github.com/apache/pulsar/pull/13075) - Allow to start bookie with Pulsar metadata store backend [#13296](https://github.com/apache/pulsar/pull/13296) - - Removed ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) + - Remove ZooKeeperClientFactory [#13303](https://github.com/apache/pulsar/pull/13303) - Use reference counting in RocksDB metadata store [#13309](https://github.com/apache/pulsar/pull/13309) - Allow configuring metadata store URL in proxy.conf [#13777](https://github.com/apache/pulsar/pull/13777) - Allow configuring metadata store URL in functions_worker.yml [#13782](https://github.com/apache/pulsar/pull/13782) @@ -48,8 +48,8 @@ - Allow configuring configurationMetadataStore when initializing cluster metadata [#13889](https://github.com/apache/pulsar/pull/13889) - Using the consistent metadata store scheme name [#13937](https://github.com/apache/pulsar/pull/13937) - Allow configuring `metadataStoreUrl` in `pulsar-perf managed-ledger` [#14145](https://github.com/apache/pulsar/pull/14145) - - Deprecated zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) - - Fixed metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) + - Deprecate zookeeper settings [#14147](https://github.com/apache/pulsar/pull/14147) + - Fix metadata cache inconsistency on doing refresh [#14283](https://github.com/apache/pulsar/pull/14283) - Support broker level dispatch rate limiter [#11325](https://github.com/apache/pulsar/pull/11325) - Support setting geo-replication clusters on topic level [#12136](https://github.com/apache/pulsar/pull/12136) - Add Multi-tiered storage key to broker.conf [#12173](https://github.com/apache/pulsar/pull/12173)