Skip to content

[run-ci][fix][flaky-test]NamespaceOwnershipListenerTests.testNamespaceBundleOwnershipListener - #11

Closed
poorbarcode wants to merge 47 commits into
masterfrom
flaky/testNamespaceBundleOwnershipListener
Closed

[run-ci][fix][flaky-test]NamespaceOwnershipListenerTests.testNamespaceBundleOwnershipListener#11
poorbarcode wants to merge 47 commits into
masterfrom
flaky/testNamespaceBundleOwnershipListener

Conversation

@poorbarcode

@poorbarcode poorbarcode commented Sep 22, 2022

Copy link
Copy Markdown
Owner

tisonkun and others added 30 commits September 19, 2022 12:25
Signed-off-by: tison <wander4096@gmail.com>

Signed-off-by: tison <wander4096@gmail.com>
…e#17723)

- The approval solution doesn't work as expected by approving the PR
  or by adding the ready-to-test label and adding a comment
  "/pulsarbot rerun-failure-checks".

- Fix the ready-to-test label check:
  - Refresh PR labels when re-running workflow
    - when re-running, the event JSON remains the same. The API
      must be used to fetch the up-to-date JSON for the PR.

- Fix the PR approval check:
  - set GITHUB_TOKEN for script so that retrieving the approval status could work
### Motivation

in the schema update, will create a `ledgerHandle` and write data to BK, after that `ledgerHandle` is no longer useful and no other object holds references to it. `ledgerHandle` will be recycled with GC, but `ledgerHandle` also hold external connections, which will cause leakage.

https://github.com/apache/pulsar/blob/40b9d7ea50cef54becb09f2543193e08375abe0b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/BookkeeperSchemaStorage.java#L452-L456

### Modifications

after the schema is updated, close the `ledgerHandle`, just like schema-read:

https://github.com/apache/pulsar/blob/40b9d7ea50cef54becb09f2543193e08375abe0b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/BookkeeperSchemaStorage.java#L519-L525
…hapter (apache#17615)

* Sidebar re-org

* Separate TLS encryption/authentication using Keystore to parent topics

* use language-specific tabs to show code snippets

* streamline headings

* Update client-libraries-java.md

* Add code snippet for python clients to support apache#17482

* Add code snippet for go clients
…ception` (apache#17700)

### Motivation

In the production environment,  we found two brokers holding the same valid locks. and one has an exceptional revalidate future with `lockBusyException`. after reading the code, there may forget the reset the cache and complete expire exception when getting lockBusyException.
… cookbook (apache#17666)

* remove duplicate file `cookbooks-encryption.md`

* Revert "remove duplicate file `cookbooks-encryption.md`"

This reverts commit a1e5a54.

* streamline encryption cookbook and make it single-sourced

* Add code snippets for Java/C++/Go clients.

* use one svg image to replace two jpg files

* client version and title updates

* improve security overview

* Add note and link for TLS encryption
…ation (apache#16917)

* [improve][txn][PIP-196] Segmented transaction buffer snapshot
Master apache#16913
### Motivation
1. Add configurations.
2. Implement the system topic client for snapshot topic and index topic.
3. Implement AbortedTxnProcessor.
4. Add transaction buffer snapshot metrics.
### Modification
Add configuration for segment snaopshot.
…on (apache#16003)

### Motivation

Now, the MongoDB source connector only supports the incremental message synchronization.
This PR adds support for full message synchronization.

Since MongDB 4.0, we can set the starting point for the change stream by the `startAtOperationTime` field.
So, we can set it to `0` to make start point the earliest.
See https://www.mongodb.com/docs/v4.2/reference/method/db.collection.watch/ for more information.
### Motivation

The current `bin/pulsar version` script only prints out the version string. When running custom builds of Pulsar, this is sometimes not enough to know which version you have. Since the `PulsarVersion` class already has more information, I propose we print out all of its information in the same format that the broker logs the information.

Here is a sample output for this specific PR:

```shell
$ bin/pulsar version
Current version of pulsar is: 2.11.0-SNAPSHOT
Git Revision 4414e3d
Git Branch print-more-version-info
Built by Michael Marshall <mmarshall@apache.org> on mmarshall-rmbp16 at 2020-01-22T09:10:15-0600
```

### Modifications

* Add more output to the `bin/pulsar version` command.

### Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

### Documentation

- [x] `doc-not-needed` 
(Please explain why)

### Matching PR in forked repository

PR in forked repository: michaeljmarshall#3
…pache#17743)

- when setup is called explicitly, cleanup should be issued before it
- remove unnecessary "resetConfig" method calls
  - resetConfig gets called as part of internalCleanup
  - when resetConfig is in cleanup, it's possible to override config before calling setup
    - this allows using setup & cleanup methods instead of using
      error prone internalSetup and internalCleanup methods which might leave something behind
* fix entryFilter stats

* fix test

* add test comment

* review fix
momo-jun and others added 11 commits September 22, 2022 08:58
### Motivation

Capturing shared ptr in the timer function will cause a circular reference.

### Modifications

- Use weak ptr instead shared ptr.
…axMessageSize (apache#17289)

### Motivation

See: apache#17188

### Modifications

Support include message header size when check maxMessageSize for cpp client
### Motivation
now `org.apache.pulsar.client.api.transaction.Transaction` dont have a interface for user to get the transaction state.

user can get the transaction state to do user's own op.
### Modifications
1. add the interface in `org.apache.pulsar.client.api.transaction.Transaction`     `getState`
2. TransactionImpl implement the interface 
```
     * Get transaction state.
     *
     * @return {@link State} the state of the transaction.
     */
    State getState();
```
### Verifying this change
add the test
@poorbarcode
poorbarcode force-pushed the flaky/testNamespaceBundleOwnershipListener branch 3 times, most recently from 160e7d0 to e962e87 Compare September 22, 2022 17:36
lordcheng10 and others added 4 commits September 22, 2022 21:06
* remove unused code

* check style

* remove unused bundleFactory

Co-authored-by: leolinchen <leolinchen@tencent.com>
…pache#17456)

 Support lower boundary shedding for ThresholdShedder (apache#17456)

The existing ThresholdShedder has the following problems, for example:
There are 11 Brokers, of which 10 are loaded at 80% and 1 is loaded at 0%.
The average load is 80 * 10 / 11 = 72.73, and the threshold to unload is 72.73 + 10 = 82.73.
Since 80 < 82.73, unload will not be trigger, and there is one idle Broker with load of 0%.

On the basis of ThresholdShedder, we adds the lower boundary judgment of the load.
When 【current usage < average usage - threshold】, the broker with the highest load will be triggered to unload
@poorbarcode
poorbarcode force-pushed the flaky/testNamespaceBundleOwnershipListener branch from e962e87 to abf6f59 Compare September 23, 2022 01:53
@poorbarcode
poorbarcode force-pushed the flaky/testNamespaceBundleOwnershipListener branch 3 times, most recently from f3fdc47 to 2617086 Compare September 23, 2022 05:59
@poorbarcode
poorbarcode force-pushed the flaky/testNamespaceBundleOwnershipListener branch 2 times, most recently from 9bb068b to ad13282 Compare September 23, 2022 08:22
@poorbarcode
poorbarcode force-pushed the flaky/testNamespaceBundleOwnershipListener branch from ad13282 to 8033180 Compare September 23, 2022 08:49
@poorbarcode
poorbarcode deleted the flaky/testNamespaceBundleOwnershipListener branch September 26, 2022 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.