Skip to content

[improve][broker] Recycle OpReadEntry in some corner cases - #16399

Merged
BewareMyPower merged 3 commits into
apache:masterfrom
BewareMyPower:bewaremypower/op-read-entry-recycle
Jul 7, 2022
Merged

[improve][broker] Recycle OpReadEntry in some corner cases#16399
BewareMyPower merged 3 commits into
apache:masterfrom
BewareMyPower:bewaremypower/op-read-entry-recycle

Conversation

@BewareMyPower

@BewareMyPower BewareMyPower commented Jul 5, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Motivation

ManagedCursorImpl maintains a field waitingReadOp as the cache of
the OpReadEntry created in asyncReadEntriesOrWait when there are no
more entries to read. However, there are two cases that the created
OpReadEntry are not recycled:

  1. asyncReadEntriesOrWait is called repeatedly when waitingReadOp is
    not null and there are no more entries. The new created OpReadEntry
    cannot pass the CAS check but it's not recycled.
  2. cancelPendingReadRequest is called. The waitingReadOp is just set
    with null and the previous reference is not recycled.

Modifications

For the two cases described above, recycle the OpReadEntry objects.

Verifying this change

Add testOpReadEntryRecycle to reproduce the corner cases and verify the
count of recycle() calls.

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@BewareMyPower BewareMyPower added this to the 2.11.0 milestone Jul 5, 2022
@BewareMyPower BewareMyPower self-assigned this Jul 5, 2022
@github-actions

github-actions Bot commented Jul 5, 2022

Copy link
Copy Markdown

@BewareMyPower Please provide a correct documentation label for your PR.
Instructions see Pulsar Documentation Label Guide.

@github-actions github-actions Bot added doc-label-missing doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Jul 5, 2022
merlimat
merlimat approved these changes Jul 5, 2022
View reviewed changes

@merlimat merlimat left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I wouldn't mark it as a bug though. It's ok to skip recycling for error condition. It wil just cause the object to be GCed instead of going back to the pool.

Comment thread managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/OpReadEntry.java Outdated
@BewareMyPower BewareMyPower changed the title [fix][broker] OpReadEntry is not recycled in some corner cases [improve][broker] OpReadEntry is not recycled in some corner cases Jul 6, 2022
@BewareMyPower BewareMyPower added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages and removed type/bug The PR fixed a bug or issue reported a bug labels Jul 6, 2022
@BewareMyPower BewareMyPower changed the title [improve][broker] OpReadEntry is not recycled in some corner cases [improve][broker] Recycle OpReadEntry in some corner cases Jul 6, 2022
eolivelli
eolivelli requested changes Jul 6, 2022
View reviewed changes
Comment thread managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/OpReadEntry.java Outdated
eolivelli
eolivelli approved these changes Jul 6, 2022
View reviewed changes

@eolivelli eolivelli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BewareMyPower
[fix][broker] OpReadEntry is not recycled in some corner cases
4f7c781 
### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

To verify all `OpReadEntry` are recycled correctly, this PRs adds two
static atomic integers to record the create count and the recycle count.
Then add `testOpReadEntryRecycle` to reproduce the corner cases and
verify these counts.
@BewareMyPower
Remove volatile
36fb2fd
@BewareMyPower
Use mockito to verify the count of recycle() calls
7232f22
@BewareMyPower
BewareMyPower force-pushed the bewaremypower/op-read-entry-recycle branch from b8d77c8 to 7232f22 Compare July 7, 2022 07:03
@BewareMyPower
BewareMyPower merged commit 6cec62e into apache:master Jul 7, 2022
@BewareMyPower
BewareMyPower deleted the bewaremypower/op-read-entry-recycle branch July 7, 2022 13:27
codelipenghui pushed a commit that referenced this pull request Jul 10, 2022
@BewareMyPower @codelipenghui
[improve][broker] Recycle OpReadEntry in some corner cases (#16399)
7c37e56 
### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

Add `testOpReadEntryRecycle` to reproduce the corner cases and verify the
count of `recycle()` calls.

(cherry picked from commit 6cec62e)
zymap pushed a commit to zymap/pulsar that referenced this pull request Jul 11, 2022
@BewareMyPower @zymap
[improve][broker] Recycle OpReadEntry in some corner cases (apache#16399
753dc07 
)

### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

Add `testOpReadEntryRecycle` to reproduce the corner cases and verify the
count of `recycle()` calls.
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jul 11, 2022
@BewareMyPower @nicoloboschi
[improve][broker] Recycle OpReadEntry in some corner cases (apache#16399
43e239a 
)

### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

Add `testOpReadEntryRecycle` to reproduce the corner cases and verify the
count of `recycle()` calls.

(cherry picked from commit 6cec62e)
(cherry picked from commit 7c37e56)
wuxuanqicn pushed a commit to wuxuanqicn/pulsar that referenced this pull request Jul 14, 2022
@BewareMyPower @weimob-wuxuanqi
[improve][broker] Recycle OpReadEntry in some corner cases (apache#16399
fac2d62 
)

### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

Add `testOpReadEntryRecycle` to reproduce the corner cases and verify the
count of `recycle()` calls.
mattisonchao pushed a commit that referenced this pull request Jul 15, 2022
@BewareMyPower @mattisonchao
[improve][broker] Recycle OpReadEntry in some corner cases (#16399)
7e17d7a 
### Motivation

`ManagedCursorImpl` maintains a field `waitingReadOp` as the cache of
the `OpReadEntry` created in `asyncReadEntriesOrWait` when there are no
more entries to read. However, there are two cases that the created
`OpReadEntry` are not recycled:
1. `asyncReadEntriesOrWait` is called repeatedly when `waitingReadOp` is
   not null and there are no more entries. The new created `OpReadEntry`
  cannot pass the CAS check but it's not recycled.
2. `cancelPendingReadRequest` is called. The `waitingReadOp` is just set
   with null and the previous reference is not recycled.

### Modifications

For the two cases described above, recycle the `OpReadEntry` objects.

### Verifying this change

Add `testOpReadEntryRecycle` to reproduce the corner cases and verify the
count of `recycle()` calls.

(cherry picked from commit 6cec62e)
@mattisonchao mattisonchao added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Jul 15, 2022
BewareMyPower added a commit that referenced this pull request Jul 29, 2022
@BewareMyPower
[improve][broker] Recycle OpReadEntry in some corner cases (#16399)
06885db 
(cherry picked from commit 6cec62e)

In addition to #16399, this PR imports the `mockito-inline` dependency
to fix the test failure.
@BewareMyPower BewareMyPower mentioned this pull request Jul 29, 2022
Merged
4 tasks
@BewareMyPower

BewareMyPower commented Jul 29, 2022

Copy link
Copy Markdown
Contributor Author

Move the release/2.8.4 label to #16869

BewareMyPower added a commit that referenced this pull request Aug 1, 2022
@BewareMyPower
[improve][broker] Recycle OpReadEntry in some corner cases (#16399)
e3391b6 
(cherry picked from commit 6cec62e)

In addition to #16399, this PR imports the `mockito-inline` dependency
to fix the test failure.
BewareMyPower added a commit that referenced this pull request Aug 2, 2022
@BewareMyPower
[branch-2.8][improve][broker] Recycle OpReadEntry in some corner cases (
123175f 
#16399) (#16869)

* [improve][broker] Recycle OpReadEntry in some corner cases (#16399)

(cherry picked from commit 6cec62e)

In addition to #16399, this PR imports the `mockito-inline` dependency
to fix the test failure.

* Use PowerMock instead of mockito-inline
@BewareMyPower BewareMyPower mentioned this pull request Sep 16, 2025
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merlimat merlimat merlimat approved these changes
@eolivelli eolivelli eolivelli approved these changes
@codelipenghui codelipenghui codelipenghui approved these changes
@Jason918 Jason918 Awaiting requested review from Jason918
@gaozhangmin gaozhangmin Awaiting requested review from gaozhangmin
@315157973 315157973 Awaiting requested review from 315157973
@congbobo184 congbobo184 Awaiting requested review from congbobo184

Assignees

@BewareMyPower BewareMyPower

Labels

area/broker cherry-picked/branch-2.9 Archived: 2.9 is end of life cherry-picked/branch-2.10 doc-not-needed Your PR changes do not impact docs release/2.9.4 release/2.10.2 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

Projects

None yet

Milestone

2.11.0

Development

Successfully merging this pull request may close these issues.

5 participants

@BewareMyPower @merlimat @eolivelli @codelipenghui @mattisonchao