I'm trying to run Dataform in "Full Refresh" mode with this Airflow Operator definition and it straight up doesn't work. It only runs the repo in non-full refresh mode. This leaves us with effectively no way to run our production DAG via "Full Refresh". I tried sorting through the code but I can't see where the arg is getting dropped.
create_workflow_invocation_full_refresh = DataformCreateWorkflowInvocationOperator(
task_id="create_workflow_invocation_full_refresh",
project_id=PROJECT_ID,
region=DATAFORM_REGION,
repository_id=REPOSITORY_ID,
workflow_invocation={
"compilation_result": "{{ task_instance.xcom_pull('create_compilation_result')['name'] }}",
"invocation_config": {
"fully_refresh_incremental_tables_enabled": True,
}
}
)
I'm trying to run Dataform in "Full Refresh" mode with this Airflow Operator definition and it straight up doesn't work. It only runs the repo in non-full refresh mode. This leaves us with effectively no way to run our production DAG via "Full Refresh". I tried sorting through the code but I can't see where the arg is getting dropped.
Class Definition:
airflow/providers/google/src/airflow/providers/google/cloud/operators/dataform.py
Line 187 in 34ed71e