Skip to content

Full Refresh arg in invocation_config does not work via DataformCreateWorkflowInvocationOperator #53843

Description

@rsenseman

I'm trying to run Dataform in "Full Refresh" mode with this Airflow Operator definition and it straight up doesn't work. It only runs the repo in non-full refresh mode. This leaves us with effectively no way to run our production DAG via "Full Refresh". I tried sorting through the code but I can't see where the arg is getting dropped.

        create_workflow_invocation_full_refresh = DataformCreateWorkflowInvocationOperator(
            task_id="create_workflow_invocation_full_refresh",
            project_id=PROJECT_ID,
            region=DATAFORM_REGION,
            repository_id=REPOSITORY_ID,
            
            workflow_invocation={
                "compilation_result": "{{ task_instance.xcom_pull('create_compilation_result')['name'] }}",
                "invocation_config": {
                    "fully_refresh_incremental_tables_enabled": True,
                }
            }
        )

Class Definition:

class DataformCreateWorkflowInvocationOperator(GoogleCloudBaseOperator):

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions