Skip to content

[Bug] airflow dag-processor -B <name> fails to resolve connection for the specified bundle #61033

@uplsh580

Description

@uplsh580

Apache Airflow version

3.1.6

If "Other Airflow 3 version" selected, which one?

No response

What happened?

In Airflow 3.1.6, the airflow dag-processor -B <BUNDLE_NAME> command fails to start because it cannot resolve the connection associated with the specified bundle.

Even though the git_conn_id is correctly defined in the metadata database and visible in the Airflow UI, the dag-processor throws an AirflowNotFoundException during the bundle's initialization phase. This prevents the processor from starting for the requested bundle.

This is error log.

$ airflow dag-processor -B bundle1
2026-01-25T09:59:38.182059Z [info     ] DAG bundles loaded: bundle1    [airflow.dag_processing.bundles.manager.DagBundlesManager] loc=manager.py:179
2026-01-25T09:59:38.183057Z [error    ] Error creating bundle 'bundle1': The conn_id `GITHUB__SAMPLE` isn't defined [airflow.dag_processing.bundles.manager.DagBundlesManager] loc=manager.py:294
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/bundles/manager.py", line 292, in get_all_dag_bundles
    yield class_(name=name, version=None, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 101, in __init__
    self.hook = GitHook(git_conn_id=git_conn_id or "git_default", repo_url=self.repo_url)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/hooks/git.py", line 68, in __init__
    connection = self.get_connection(git_conn_id)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/hook.py", line 61, in get_connection
    conn = Connection.get(conn_id)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/definitions/connection.py", line 225, in get
    return _get_connection(conn_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/context.py", line 174, in _get_connection
    raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `GITHUB__SAMPLE` isn't defined
Bundles not found: bundle1

Interestingly, the command works perfectly fine and resolves all connections as expected when running a plain airflow dag-processor without any flags.

What you think should happen instead?

No response

How to reproduce

  1. Configure a DAG bundle in airflow.cfg (e.g., named bundle1) using a GitBundle or any bundle that requires a connection.
    [dag_processor]
    dag_bundle_config_list = [{"classpath":"airflow.providers.git.bundles.git.GitDagBundle","kwargs":{"git_conn_id":"GITHUB__SAMPLE","refresh_interval":60,"subdir":"dags","tracking_ref":"main"},"name":"bundle1"}]
  2. Ensure the connection GITHUB__SAMPLE is created and exists in Airflow.
  3. Run the dag-processor for this specific bundle:
    airflow dag-processor -B bundle1

Operating System

Airflow official docker image (apache/airflow:3.1.6)

Versions of Apache Airflow Providers

Providers info
apache-airflow-providers-amazon           | 9.19.0
apache-airflow-providers-celery           | 3.15.0
apache-airflow-providers-cncf-kubernetes  | 10.12.0
apache-airflow-providers-common-compat    | 1.11.0
apache-airflow-providers-common-io        | 1.7.0
apache-airflow-providers-common-messaging | 2.0.1
apache-airflow-providers-common-sql       | 1.30.2
apache-airflow-providers-docker           | 4.5.1
apache-airflow-providers-elasticsearch    | 6.4.2
apache-airflow-providers-fab              | 3.1.1
apache-airflow-providers-ftp              | 3.14.0
apache-airflow-providers-git              | 0.2.0
apache-airflow-providers-google           | 19.3.0
apache-airflow-providers-grpc             | 3.9.1
apache-airflow-providers-hashicorp        | 4.4.1
apache-airflow-providers-http             | 5.6.2
apache-airflow-providers-microsoft-azure  | 12.10.1
apache-airflow-providers-mysql            | 6.4.0
apache-airflow-providers-odbc             | 4.11.0
apache-airflow-providers-openlineage      | 2.9.2
apache-airflow-providers-postgres         | 6.5.1
apache-airflow-providers-redis            | 4.4.1
apache-airflow-providers-sendgrid         | 4.2.0
apache-airflow-providers-sftp             | 5.5.1
apache-airflow-providers-slack            | 9.6.1
apache-airflow-providers-smtp             | 2.4.1
apache-airflow-providers-snowflake        | 6.8.1
apache-airflow-providers-ssh              | 4.2.1
apache-airflow-providers-standard         | 1.10.2

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions