Skip to content

Make field on Incremental tables default to false as it was before. - #1942

Merged
Ceridan merged 2 commits into
mainfrom
fix-full-refresh-mode
Mar 25, 2025
Merged

Make field on Incremental tables default to false as it was before.#1942
Ceridan merged 2 commits into
mainfrom
fix-full-refresh-mode

Conversation

@Ceridan

@Ceridan Ceridan commented Mar 25, 2025

Copy link
Copy Markdown
Contributor

Configs refactoring introduced by #1780 changes the default behavior of protected field on Incremental tables, see #1914

The current PR changes the default behavior to make it consistent with what we had before:

  1. If config has protected field set, then the value of the field will be used.
  2. If config omits protected field, it will be set to false.
  3. JavaScript API will be slightly changed: by default, protected will be set to false.
  4. If in JavaScript API .protected() will be called without arguments (like it was possible before) then the field will be set to through.
  5. If in JavaScript API .protected() will be called with argument (true or false) then the field will be set to the value of the argument passed.

The logic is quite complicated here but we are trying to mimic the previous behavior to preserve backward compatibility.

Closes #1914

@Ceridan
Ceridan requested a review from a team as a code owner March 25, 2025 10:41
@Ceridan
Ceridan requested review from DiasDauletov and kolina and removed request for a team and DiasDauletov March 25, 2025 10:41
Comment thread core/main_test.ts Outdated
@Ceridan
Ceridan merged commit cac43b2 into main Mar 25, 2025
@Ceridan
Ceridan deleted the fix-full-refresh-mode branch March 25, 2025 11:32
@Ceridan Ceridan mentioned this pull request Mar 25, 2025
potiuk pushed a commit to apache/airflow that referenced this pull request Aug 1, 2026
The reverted change was meant to fix #53843 (invocation_config believed
dropped when workflow_invocation is passed as a dict), but it is a
wire-identical no-op: proto-plus already converts nested dicts
recursively, and the pre- and post-change requests serialize
byte-identically, so the dict path never dropped the config. The
reported symptom came from dataform-core 3.0.10-3.0.16 defaulting
incremental tables to protected=true (silently disabling full refresh
server-side; fixed upstream in core 3.0.17 via
dataform-co/dataform#1942). #53843 is closed as not planned. The
accompanying test only pinned third-party proto-plus conversion
behavior, which our testing standards exclude.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Queries fail to run in full refresh mode

2 participants