Skip to content

fix compaction entry read exception#11175

Merged
eolivelli merged 2 commits into
apache:masterfrom
hangc0276:chenhang/fix_compaction_entry_read_exception
Jul 2, 2021
Merged

fix compaction entry read exception#11175
eolivelli merged 2 commits into
apache:masterfrom
hangc0276:chenhang/fix_compaction_entry_read_exception

Conversation

@hangc0276

Copy link
Copy Markdown
Contributor

Motivation

When read entry from compacted ledger, it will get the following exception when read the end of compacted ledger

12:04:11.193 [broker-topic-workers-OrderedScheduler-7-0] ERROR org.apache.bookkeeper.client.LedgerHandle - IncorrectParameterException on ledgerId:3753 firstEntry:-4276948922 lastEntry:-4276948822
12:04:11.194 [broker-topic-workers-OrderedScheduler-7-0] ERROR org.apache.pulsar.broker.service.persistent.PersistentDispatcherSingleActiveConsumer - [persistent://public/default/__change_events-partition-0 / multiTopicsReader-5358dc9a38-Consumer{subscription=PersistentSubscription{topic=persistent://public/default/__change_events-partition-0, name=multiTopicsReader-5358dc9a38}, consumerId=3, consumerName=fd550, address=/202.168.117.36:45676}] Error reading entries at 3837:0 : java.util.concurrent.CompletionException: org.apache.bookkeeper.client.BKException$BKIncorrectParameterException: Incorrect parameter input - Retrying to read in 15.0 seconds

The reason is when startPoint == NEWER_THAN_COMPACTED it will set cursor to normal ledger to read the following entry and should just complete the read cycle and return.

if (startPoint == NEWER_THAN_COMPACTED) {
cursor.seek(compactionHorizon.getNext());
callback.readEntriesComplete(Collections.emptyList(), ctx);
}
return readEntries(context.ledger, startPoint, endPoint)
.thenAccept((entries) -> {
Entry lastEntry = entries.get(entries.size() - 1);
cursor.seek(lastEntry.getPosition().getNext());
callback.readEntriesComplete(entries, ctx);
});

Modification

  1. When read the end of compacted ledger, just return this read cycle.

@codelipenghui codelipenghui left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me, is it able to add a test for avoiding the regression?

@eolivelli
eolivelli merged commit c716495 into apache:master Jul 2, 2021
codelipenghui pushed a commit that referenced this pull request Jul 4, 2021
@codelipenghui codelipenghui added the cherry-picked/branch-2.7 Archived: 2.7 is end of life label Jul 4, 2021
codelipenghui pushed a commit that referenced this pull request Jul 7, 2021
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Jul 7, 2021
eolivelli pushed a commit to eolivelli/pulsar that referenced this pull request Feb 25, 2022
(cherry picked from commit c716495)
(cherry picked from commit c710c18)
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-picked/branch-2.7 Archived: 2.7 is end of life cherry-picked/branch-2.8 Archived: 2.8 is end of life release/2.7.3 release/2.8.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants