Skip to content

[Feature] Avoid logging full rows with sensitive information on conversion failure in Flink CDC #4290

@atallahade

Description

@atallahade

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Using a Flink CDC connector, when a row fails to convert according to the given schema, Paimon logs the entire row, which may contain sensitive information.
You can see this in CdcRecordUtils.java. For example:

2024-10-08 14:14:48,673 [] INFO  org.apache.paimon.flink.sink.cdc.CdcRecordUtils              [] - Failed to convert value <REDACTED_ROW> to type <REDACTED_SCHEMA>. Waiting for schema update.

Solution

Logging only the primary key could be a solution.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions