Skip to content

[improve][txn][PIP-196]Segmented transaction buffer snapshot configuration - #16917

Merged
liangyepianzhou merged 5 commits into
apache:masterfrom
liangyepianzhou:xiangying/pip/pip196-1
Sep 20, 2022
Merged

[improve][txn][PIP-196]Segmented transaction buffer snapshot configuration#16917
liangyepianzhou merged 5 commits into
apache:masterfrom
liangyepianzhou:xiangying/pip/pip196-1

Conversation

@liangyepianzhou

@liangyepianzhou liangyepianzhou commented Aug 2, 2022

Copy link
Copy Markdown
Contributor

Master Issue: #16913

Motivation

  1. Add configurations.
  2. Implement system topic client for snapshot topic and index topic.
  3. Implement AbortedTxnProcessor.
  4. Add transaction buffer snapshot metrics.

Modification

Add configuration

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API: (yes / no)
  • The schema: (yes / no / don't know)
  • The default values of configurations: (yes / no)
  • The wire protocol: (yes / no)
  • The rest endpoints: (yes / no)
  • The admin cli options: (yes / no)
  • Anything that affects deployment: (yes / no / don't know)

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@github-actions

github-actions Bot commented Sep 3, 2022

Copy link
Copy Markdown

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions Bot added the Stale label Sep 3, 2022
Master #16913
### Motivation
1. Add configurations.
2. Implement system topic client for snapshot topic and index topic.
3. Implement AbortedTxnProcessor.
4. Add transaction buffer snapshot metrics.
### Modification
Add configuration
@liangyepianzhou liangyepianzhou changed the title [Improve][Txn]TransactionBuffer multiple-snapshot configuration [improve][txn][PIP-196]Segmented transaction buffer snapshot configuration Sep 6, 2022
@liangyepianzhou

Copy link
Copy Markdown
Contributor Author

@codelipenghui @congbobo184 @gaoran10 Please help review this PR when you have time. THX.
@momo-jun Please help review this PR from a technical writer.

Comment thread conf/broker.conf Outdated
Comment thread conf/broker.conf Outdated
Comment thread conf/broker.conf Outdated
Comment thread conf/broker.conf Outdated
Comment thread conf/broker.conf Outdated
@github-actions github-actions Bot removed the Stale label Sep 7, 2022
@congbobo184

Copy link
Copy Markdown
Contributor

@liangyepianzhou

Copy link
Copy Markdown
Contributor Author

when https://lists.apache.org/thread/bqoy3oz8flvxy7xpmnw81cr4c9sz5vy0 +3 binding can merge

It already has +3 binding.

@codelipenghui codelipenghui added this to the 2.12.0 milestone Sep 19, 2022
@codelipenghui codelipenghui added area/transaction type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages labels Sep 19, 2022
Comment thread conf/broker.conf Outdated
Comment thread conf/standalone.conf Outdated
@codelipenghui

Copy link
Copy Markdown
Contributor

/pulsarbot run-failure-checks

@codelipenghui codelipenghui reopened this Sep 20, 2022
@liangyepianzhou
liangyepianzhou merged commit f6665fb into apache:master Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/transaction doc-not-needed Your PR changes do not impact docs ready-to-test type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants