KCA doesn't handle unchecked ConnectException/KafkaException for the task, it may lead to the connector hanging - #12441
Merged
Merged
Conversation
…n for the task, it may lead to the connector hanging
dlg99
changed the title
eolivelli
requested review from
codelipenghui,
congbobo184,
gaoran10 and
jerrypeng
October 21, 2021 12:01
2 tasks
congbobo184
reviewed
Oct 22, 2021
|
|
||
| @Override | ||
| public void commit() throws InterruptedException { | ||
| throw new org.apache.kafka.connect.errors.ConnectException("blah"); |
Contributor
Author
There was a problem hiding this comment.
@congbobo184 ErrFileStreamSourceTask is used in the test KafkaConnectSourceErrTest; this mimics the behavior encountered with debezium connector (commit throws ConnectException). The test used to hang at the kafkaConnectSource.read() call without the fix.
congbobo184
approved these changes
Oct 22, 2021
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Oct 22, 2021
…n for the task, it may lead to the connector hanging (apache#12441) (cherry picked from commit 34f237b)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Oct 22, 2021
…n for the task, it may lead to the connector hanging (apache#12441) (cherry picked from commit 34f237b)
codelipenghui
pushed a commit
that referenced
this pull request
Oct 24, 2021
### Motivation Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. #12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. ### Modifications Closing offset store on connector stop.
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Oct 25, 2021
### Motivation Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. apache#12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. ### Modifications Closing offset store on connector stop. (cherry picked from commit 63454e9)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Oct 25, 2021
Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. apache#12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. Closing offset store on connector stop. (cherry picked from commit 63454e9)
hangc0276
pushed a commit
that referenced
this pull request
Nov 4, 2021
Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. #12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. Closing offset store on connector stop. (cherry picked from commit 63454e9)
eolivelli
pushed a commit
that referenced
this pull request
Nov 9, 2021
### Motivation Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. #12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. ### Modifications Closing offset store on connector stop. (cherry picked from commit 63454e9)
eolivelli
pushed a commit
to eolivelli/pulsar
that referenced
this pull request
Nov 29, 2021
…n for the task, it may lead to the connector hanging (apache#12441)
eolivelli
pushed a commit
to eolivelli/pulsar
that referenced
this pull request
Nov 29, 2021
### Motivation Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. apache#12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. ### Modifications Closing offset store on connector stop.
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Mar 1, 2022
…n for the task, it may lead to the connector hanging (apache#12441) (cherry picked from commit 34f237b) (cherry picked from commit 8dea755)
merlimat
pushed a commit
to apache/pulsar-connectors
that referenced
this pull request
Mar 26, 2026
### Motivation Source connectors based on KCA (all debezium ones) don't stop properly on error / don't restart. apache/pulsar#12441 fixes one problem, this PR fixes another: ofsetStore is not closed on connector stop() and producer/consumer aren't closed too, preventing the connector from shutting down. ### Modifications Closing offset store on connector stop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The source can throw unchecked KafkaException/ConnectException.
This is not handled properly and may result in hanging connector, as experienced in prod with Debezium Postgres connector.
Modifications
Added unit test to repro the issue and fixed it.
Also now KCA Source will throw exception in read() if the flush failed. This will allow connector fail fast and restart instead of doing something while the task cannot flush/
Verifying this change
This change added unit test
Does this pull request potentially affect one of the following parts:
NO
If
yeswas chosen, please highlight the changesDocumentation
no-need-doc