Circular import issues with airflow_local_settings.py

[dstandish]

Our docs on dag policy demonstrate usage with type annotations:
https://airflow.apache.org/docs/apache-airflow/stable/concepts.html?highlight=dag_policy#dag-level-cluster-policy

def dag_policy(dag: DAG):
    """Ensure that DAG has at least one tag"""
    if not dag.tags:
        raise AirflowClusterPolicyViolation(
            f"DAG {dag.dag_id} has no tags. At least one tag required. File path: {dag.filepath}"
        )

The problem is, by the time you import DAG with from airflow import DAG, airflow will have already loaded up the settings.py file (where processing of airflow_local_settings.py is done), and it seems nothing in your airflow local settings file gets imported.

So none of these examples would actually work.

To test this you can add a local settings file containing this:

from airflow import DAG

raise Exception('hello')

Now run airflow dags list and observe that no error will be raised.

Remove the DAG import. Now you'll see the error.

Any suggestions how to handle this?

There are a couple conf settings imported from settings. We could move these to airflow/__init__.py. But less straightforward would be what to do about settings.initialize(). Perhaps we could make it so that initialized is called within settings?

[kaxil]

Do you see the same error when using from airflow.models.dag import DAG ?

[dstandish]

Nope!

Here's how to test:

airflow_local_settings.py:

#  from airflow.models import DAG
#  from airflow import DAG
def dag_policy(dag):
    dag.template_searchpath = ['hello']

in dag file:

from airflow import settings
settings.dag_policy(dag)
print(dag.template_searchpath)

When you import from models, the dag policy works. When you import from airflow it does not.

So I guess this is mostly a non issue.

One thing we can do is fix this line. I can make a PR for that. Do you think it's worth adding any documentation?