PS-11080 fix: Spoiled logical clock information in rewritten binlog (part 3)#128
Open
percona-ysorokin wants to merge 2 commits into
Open
PS-11080 fix: Spoiled logical clock information in rewritten binlog (part 3)#128percona-ysorokin wants to merge 2 commits into
percona-ysorokin wants to merge 2 commits into
Conversation
…part 2) https://perconadev.atlassian.net/browse/PS-11080 This is a prerequisite commit required to implement rewriting 'sequence_number' / 'last_committed' / 'transaction_length' in GTID events. Along with last seen transaction GTID, 'binstv::events::reader_context' now also keeps track of the last seen GTID event sequence_number. At the same time 'binsrv::storage' expects one extra 'transaction_sequence_number' argument passed to its 'write_event()' method (which is expected to be taken from the 'binstv::events::reader_context::get_transaction_sequence_number()'). It implements the same logic of storing transaction's sequence_number for both "ready to be flushed" buffered event data and for the "incomplete transaction" buffered event data as for 'timestamps' and 'gtids'. Binlog metadata file extended with one more value 'last_sequence_number' that holds the 'sequence_number' value of the last GTID event (either GTID_LOG, ANONYMOUS_GTID_LOG or GTID_TAGGED_LOG events) written to the binlog file. Co-authored-by: Yura Sorokin <yura.sorokin@percona.com>
…part 3) https://perconadev.atlassian.net/browse/PS-11080 Implemented generic event rewriting mechanism ('binsrv::events::rewriter::rewrite()' static method) that performs the following operations. * For GTID_LOG /ANONYMOUS_GTID_LOG it changes 'sequence_number' / 'last_committed' fields in the post header and relocates the event (changes 'next_event_position' in the common header). * For GTID_TAGGED_LOG it changes 'sequence_number' / 'last_committed' in the serializer-encoded event body, which because of the variable length encoding of individual elements in the archive may change its size and may require updating 'transaction_length' in the event body and 'event_size' in the common header. In addition, it also performs event relocation. * For every other event it simply performs event relocation. All these changes are performed under a special guard ('binsrv::events::event_updatable_view::write_proxy') which guarantees that the event checksum will be recalculated / added upon finalizing all field value updates. We also make sure that all rewritten events will have a footer with properly calculated checksum (event if it was not present in the original event). The logic for updating 'sequence_number' / 'last_committed' in GTID events is encapsulated inside ('fix_sequence_number_and_last_committed()' function). Refactored the way how several event fields can be modified in a single run via 'event_updatable_view' with guarantee that event's checksum will be properly recalculated. Introduced 'rewriter::generic_materialize()' template function that accepts generic field modification functor. Also introduced two simplified helper functions (which use the generic one underneath) for the most typical cases: * 'rewriter::materialize()' * 'rewriter::materialize_and_relocate()' Co-authored-by: Kamil Holubicki <kamil.holubicki@percona.com>
percona-ysorokin
requested review from
Copilot and
kamil-holubicki
and removed request for
Copilot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://perconadev.atlassian.net/browse/PS-11080
Implemented generic event rewriting mechanism
('binsrv::events::rewriter::rewrite()' static method) that performs the following
operations.
'last_committed' fields in the post header and relocates the event (changes
'next_event_position' in the common header).
serializer-encoded event body, which because of the variable length encoding
of individual elements in the archive may change its size and may require
updating 'transaction_length' in the event body and 'event_size' in the common
header. In addition, it also performs event relocation.
All these changes are performed under a special guard
('binsrv::events::event_updatable_view::write_proxy') which guarantees that
the event checksum will be recalculated / added upon finalizing all field value
updates. We also make sure that all rewritten events will have a footer with
properly calculated checksum (event if it was not present in the original event).
The logic for updating 'sequence_number' / 'last_committed' in GTID events
is encapsulated inside ('fix_sequence_number_and_last_committed()' function).
Refactored the way how several event fields can be modified in a single run via
'event_updatable_view' with guarantee that event's checksum will be properly
recalculated. Introduced 'rewriter::generic_materialize()' template function that
accepts generic field modification functor. Also introduced two simplified helper
functions (which use the generic one underneath) for the most typical cases: